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ABSTRACT 

This document reports on a focus group meeting 
convened by the National Association of State Directors of Special 
Education (NASDSE) Board of Directors to examine the concept of 
educational accountability for all students, including students with 
disabilities. The outcome of the meeting was a definition of 
"accountability" and a conceptual model of educational 

accountability, which has its theoretical basis in the concept of the 
"social process triangle" and which includes three components: (1) 
inputs and processes (to guarantee eaucational equity), (2) system 
results (to guarantee program effectiveness), and (3) individual 
student learning (to guarantee individual student achievement). The 
current unbalanced status of educational accountability is described. 
The report concludes that a process must be undertaken to regain the 
balance in accountability with the cooperative involvement of all 
relevant constituencies. Most of the document consists of background 
materials, including the following papers: "Learner-Based 
Accountability: Making Schools Work for All Students" (Mary Ann 
Lachat); "State Compliance Monitoring Practices: An Update" (Eileen 
M . Ahearn) : "A Framework for Accountability: Concepts. Approaches, 
and Issues" (Edward J. McCaul); "State Education Accountability 
Reports, Indicator Reports, and Report Cards" (Rolf K. Blank); and 
"Don't Test, Don't Tell" (Bill Zlatos). Model diagrams complete the 
report. (DB) 
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This document is a report of a focus group meeting convened by the National 
Association of State Directors of Special Education (NASDSE) Board of Directors in 
February, 1995, to examine the concept of educational accountability for all students including 
students with disabilities. Meeting participants included the eight State Directors of Special 
Education who comprise the NASDSE Board, six additional State Directors each representing 
a different region of the country, NASDSE staff, and a facilitator. 

The outcome of the meeting is a conceptual model of educational accountability that 
includes three components: inputs and processes, system results, and individual student 
learning. The report explains the theoretical basis of the model, the component parts, and the 
ways in which each element is related to each other element. The current unbalanced status 
of educational accountability is also described. 

The report concludes that a process must be undertaken to regain the balance in 
accountability with the cooperative involvement of all relevant constituencies. 
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FOREWORD 



This report is the result of a study done under Project FORUM, a contract funded by 
the Office of Special Education Programs of the U. S. Department of Education and located 
at the National Association of State Directors of Special Education (NASDSE). Project 
FORUM carries out a variety of activities that provide information needed for program 
improvement, and promote the utilization of research data and other information for 
improving outcomes for students with disabilities. The project also provides technical 
assistance and information on emerging issues, and convenes small work groups to gather 
expert input, obtain feedback, and develop conceptual frameworks related to critical topics 
in special education. 

This activity is part of a series of Project FORUM tasks related to the topic of 
compliance monitoring in special education. The purpose of this brief analysis is to document 
the substance of a focus group meeting held by the National Association of State Directors 
of Special Education to design the initial structure of an articulated model of accountability 
in education. It was planned in response to a perceived need to reconsider the monitoring 
process in special education and to incorporate responsibility for the progress and outcomes 
of students with disabilities into a comprehensive accountability system for education. 
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REPORT OF A FOCUS GROUP ON ACCOUNTABILITY 



Background and Purpose of the Meeting: 

The focus group meeting was planned and conducted by the Board of Directors of the 
National Association of State Directors of Special Education (NASDSE). Support for the 
meeting was provided by Utah State Uni versity through the efforts of Glen Latham, Director 
of the Mountain Plains Regional Resource Center (MPRRC). A copy of the agenda is 
included in this report as Appendix A. 



Participants: In addition to the NASDSE Board of Directors who convened the 
meeting, participants included one State Director representing each Regional Resource Center, 
the NASDSE Executive Director, a Project FORUM staff member, and the meeting facilitator. 
The list is as follows: 

State Directors of Special Education: 

Bill East, AL Pam Kaufmann, MA 

Diane Sydoriak, AR David Stockford, ME 

Kay Lund, AZ Richard Baldwin, MI 

Fred Smokoski, CO Wayne Erickson, MN 

Martha Brooks, DE John Corpolongo, OK 

Gail Lieberman, EL Jill Gray, TX 

Hal Hayden, KY Steve Kukic, UT 



Other Participants: 

Martha Fields, NASDSE 

Glen Latham, Utah State University & MPRRC 

Mae Taylor, UT Department of Education 

Marilyn Crocker, Facilitator 

Eileen Ahearn, Project FORUM at NASDSE 
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Purpose of the Meeting: 



The overall goal of the focus group was to develop a shared vision of accountability. 
This goal was clarified in pre-meeting documents in terms of six anticipated outcomes: 

1 . A shared understanding of the differences among input/process accountability, 

results-based accountability, and the accountability for individual student 
performance and due process; 

2. A shared understanding of the need for a multi-component and multi-level 

articulated model of accountability; 

3. A shared understanding of the need for multiple stakeholder roles and 

responsibilities in the implementation of a comprehensive articulated 
model of accountability; 

4. A shared understanding of the relationship between an accountability system 

for the education of students with disabilities, and the broad-based 
systems of accountability that undergird states' educational reform 
efforts; 

5. Initial ideas about how best to report nationally the status of the education of 

students with disabilities so that the creation of a unified system of 
accountability is prompted; and, 

6. A plan for sharing and refining the initial understandings and vision that are 

developed as a result of this meeting with other stakeholders. 

The meeting was conceived as the next step in a process begun with the following 

efforts: 

A meeting on monitoring convened by Dr. Tom Hehir in January, 1994 that 
engaged key stakeholders in an initial discussion related to the federal 
monitoring process; 

The National Agenda for Achieving Better Results for Children and Youth with 
Disabilities developed in June, 1994 that developed a vision statement 
that called for strategic actions to implement accountability systems that 
monitor program effectiveness, balancing process with results; and, 

The Third National Monitoring Conference held in November, 1994 at which 
information was shared on the status of both federal monitoring of states 
and state monitoring of school districts. 
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Background Materials: 

Prior to the meeting, participants were provided with some materials related to the 
topic of accountability and a preliminary version of an articulated model. The background 
documents (incorporated as Appendix B of this document) included: 

A briefing paper for the Accountability Focus Group that includes a copy of 

the preliminary model for consideration at the meeting; 
Notes from Tom Hehir's presentation at the Third National Monitoring 

Conference in November, 1994; 
A monograph prepared for the Northeast Regional Resource Center by Mary 

Ann Lachat entitled, Learner-Based Accountability: Making Schools 

Work for All Students. 
A draft report from Project FORUM entitled State Compliance Monitoring 

Practices: An Update; 
A 1993 paper by Edward McCaul entitled A Framework for Accountability: 

Concepts, Approaches, and Issues 
o An inventory of state accountability reports prepared by the Council of Chief 

State School Officers entitled State Education Accountability Reports, 

Indicator Reports, and Report Cards; 
A brief summary of the new accountability structure for Title I under the new 

Improving America's Schools Act; and, 
A copy of ihe article, "Don't Test, Don't Tell", from the November, 1994 

issues of The American School Board Journal. 



Process of the Meeting: 

After introductory remarks by Gail Lieberman covering the puipose and goals of the 
meeting, the facilitator led a discussion that covered each participant's expected outcomes of 
the meeting and an initial set of ground rules for subsequent sessions. 

The first topic of discussion was a definition of accountability. Then, the facilitator 
presented a conceptual framework for the development of the draft model of accountability. 
Using the framework, the participants spelled out the three primary components of the draft 
model of accountability, discussed the profound function of each, and identified a list of 
examples to illustrate each component. 

The major emphasis of the second day was a detailing of the dynamical relationships 
between and among the three components of the accountability model using a combination 
of large group and individual work. A set of specific action steps was devised and placed on 
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a calendar. Specific assignments were made for sharing the approach with various 
constituencies, starting with a scheduled meeting between the NASDSE Executive Director 
and Tom Hehir, Director of the Office of Special Education Programs of the U.S. Department 
of Education. Plans were made to obtain input from various groups on the model as a first 
step in the process of movement toward the balance represented by the vision. This action 
plan was reviewed and finalized by the NASDSE Board before the end of the meeting. 

The meeting closed with a discussion about the meeting outcomes and a commitment 
to the action plan. 



OUTCOMES OF THE MEETING 
Accountability in Education 

Consensus was reached on the meaning of accountability in education as a result of 
focus group discussions and the vision from the National Agenda. The following statement 
represents that consensus: 

Accountability is the process by which we take account of what 
we intend; a measuring and diagnosing; being answerable for 
something; a way of ensuring that children are making progress 
toward appropriate outcomes, both cognitive and non-cognitive. 

Educators are accountable to families and parents, elected officials, and to the public 
in general. 

The conceptual framework chosen to represent accountability is based on a dynamic 
balance between and among the three major components of the accountability system. The 
pivotal concept for the model is described as follows: 

The vision for balanced accountability is an educational system 
which is accountable for ensuring that all children, including 
those with disabilities, benefit form their educational experience 
through equal access, high standards and high expectations, and 
become caring, productive, socially involved citizens who are 
committed to life-long learning. 

A model of accountability based on these concepts was developed during the remainder 
of the meeting. It is illustrated on the next page and discussed in the following section. 
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Conceptual Framework of the Model 

The conceptual framework used to devise the model of accountability is known as the ® 
social proc ess triangle described in the book, The Social Dynamics o/Humanness, issued by 
the Institute of Cultural Affairs of Chicago in 1970. This model involves viewing a social 
process in terms of its three fundamental components: its economic or foundational aspect; 
its political or organizational aspect; and its cultural or meaning-giving aspect. In an ideal 
state, each of the three poles of the triangle is robust and performs its unique function to ® 
provide balance. Often in reality, however, a relative imbalance prevails with one pole often 
assuming dominance and functioning as a kind of "tyrant," a second pole supporting the 
dominant one in an "ally" relationship, and the third pole manifesting a "collapsed" state. 

Once the dysfunction has been recognized, achieving balance requires re-empowering ® 
the collapsed pole. This can be accomplished by working directly to revitalize its functions, 
and revitalization is often accomplished by re-directing the ally away from supporting the 
dominant role. In order to move toward balance, it is helpful to understand the "profound 
function" or unique contribution of each component as well as the nature of the relationships 
between and among the components. @ 

This theoretical framework was used by the focus group participants to develop a 
dynamic model of educational accountability. 



Components of the Educational Accountability Model 

In terms of educational accountability, the three components of the model that must 
be balanced and the unique function of each is as follows: 

Inputs and Processes guarantees educational equity. Examples of this 
component include (but are not limited to) the following: EEP; LRE; 
access standards; procedural safeguards; parent involvement; staffing 
credentials; finance/funding; staff development; policy development; 
diversity; IDEA; MIS; demand for use of effective practice. ® 

System Results guarantees program effectiveness. Examples of this component 
include (but are not limited to) the following: standards; blended 
system; curriculum; ongoing measurement; multiple measures of system 
effectiveness; continuous improvement including baseline data, ® 
longitudinal studies, multiple indicators such as dropouts, retention, 
completion; cognitive and non-cognitive; staff development; flexibility 



Ri.port of Accountability Focus Group Meeting p aRe 6 

Project FORUM at NASDSE, Task U7-3-4 March 2 3, 7995 

14 

ERIC 



and accessibility to program; appropriate reinforcements, sanctions, 
incentives; school report cards; MS; state, district, school strategic plan. 

0 Individual Student Learning guarantees individual student achievement. 

Examples of this component include (but are not limited to) the 
following: parent involvement; teacher empowerment; IEP; 
academic/non-academic standards; individual expectati :ns both cognitive 
and non-cognitive; multiple measures of individual student progress;' 
MIS; flexibility in assessment modes; continuous progress with 
benchmarks along the way. 

The relational dynamics among and between these three components of educational 
accountability provide the potential for a balanced system when each component functions 
in a robust manner, and there is no expansion of any one of the elements of accountability 
to the impairment of any other element. (See Appendix C for diagrams of the model as 
balanced and unbalanced.) 

A study of accountability in education, especially as it relates to special education, 
reveals the following current status: 

0 the inputs and processes leg of the triangle, in the form of compliance 
monitoring, has usurped the entire function of accountability thereby 
becoming a tyrant in terms of this model; 

0 the system results component of accountability has acquiesced to that tyrant as 
an ajly_ by allowing the exclusion of students with disabilities from 
district assessments; and, 

° there has been no accountability for individual student outcomes leaving no 
role for the collapsed accountability component of individual student 
learning. 

The appropriate dynamic relationship of any one component to each of the other 
components can be described in terms of three basic relational categories: what that 
component creates for each of the other components, what it limits for each of the other 
components, and what it sustains for each other component. A review of these functions 
portrays the essential interrelationship among the components that interact on each other to 
establish and maintain balance. A few examples of the factors that constitute these relational 
dynamics for the educational accountability model were developed and are presented in the 
next section. 
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Dynamic Relationships Within the Model 

The following is a list of examples of the interrelationships among all the components of 
the dynamic model of educational accountability: 

A) The relationship enacted by the first component, Rights, Inputs and Processes, 
in reference to the second component, System Results is as follows: 

It creates: It provides reliable, valid data to inform system 

configuration. 

It limits: It defines boundaries for opportunities and flexibility. 
It sustains: It reinforces/renews resources for systems. 

The relationship enacted by the first component, Rights, Inputs and Processes, 
in referenc? to the third component, Individual Student Learning is as follows: 

It creates: It establishes tools necessary to support and facilitate learning. 
It limits: It requires opportunity for each and all. 
It sustains: It strengthens emphasis on individualization and parent 

involvement. 



B) The relationship enacted by the second component. System Results, in reference 
to theirs? component, Rights, Inputs and Processes, is as follows: 

It creates: It shapes inputs required to achieve results. 
It limits: It demands constant renewal of inputs/process measures. 
It sustains: It ensures continuity and consistency of inputs across 
populations. 

The relationship enacted by the second component, System Results, in reference 
to the third component, Individual Student Learning, is as follows: 

It creates: It demands high expectations for teaching and learning. 

It limits: It requires connection between what we know about a child and 

instructional strategies. 
It sustains: It strengthens emphasis on individual learning. 
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C) The relationship enacted by the third component, Individual Student Learning, 
m reference to the first component, Fights, Inputs and Processes, is as follows: 



It creates: It derives the inputs/processes (creates context and purpose). 

It limits: It demands individualized instruction. 

It sustains: It supports continuous improvement and ongoing match. 

The relationship enacted by the third component, Individual Student Learning, 
in reference to the second component, System Results, is as follows: 

It creates: It forces the system to assume ownership for all children. 
It limits: It guards against over-generalization of system results. 
It sustains: It validates system standards. 



CONCLUSION 

A model of dynamically balanced accountability for education is obviously complex, 
requiring shared roles and responsibilities of multiple stakeholder groups in many different 
configurations including local, state and national, as well as classroom, school and district 
levels. Such stakeholder groups include — but are not limited to — the following: children and 
youth, parents and families, advocates, general and special education administrators, general 
and special education teachers and specialists, the business community, legislators, boards of 
education, higher education, associations, and other service agencies. 

It is the responsibility of all relevant constituencies working individually ard together to 
establish and maintain a balanced approach to educational accountability. The first step in 
this process must be a complete review of compliance monitoring, putting it into an 
appropriate perspective at the federal and state and local levels while, at the same time., 
moving toward the complete incorporation of students with disabilities into evaluations of 
school system results. Initial steps taken in the last few years to assess and improve 
individual outcomes for students with disabilities is a third — and equally critical — component 
of the strategy needed to achieve a balanced approach to educational accountability. It is only 
through an exhaustive review and revision of every component of accountability that a new 
paradigm can be constructed that will provide a balanced and complete portrait of the 
performance of public education in the United States. 
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AGENDA 

FOCUS GROUP TO EXPLORE AN ARTICULATED MODEL OF ACCOUNTABILITY 
FOR THE EDUCATION OF STUDENTS WITH DISABILITIES 

February 24-26, 1995 



FRIDAY : 

1:00-1:15 Welcome and Introductions: Martha Fields 

1:15-1:45 Purpose and Overview: Gail Lieberman 

1:45 - 3:00 Background: Marilyn Crocker 

A. Consensus on meeting outcomes 

B. Agreement about meeting guidelines and ground rules 

C. Review and discussion of background reading materials 

3:00-3:15 BREAK 

3:15 - 6:00 Task One: Clarifying the Components of the Model 

{This activity will take place is a full group session} 

A. What are the key elements of the model? 

B. What are the functions of each element? 

C. What is the unique contribution of each element? 



7:00 

SATURDAY 
8:00 - 8:30 
8:30 - 9:00 



DINNER 



BREAKFAST 

Framing the task for today: 



Task Two: Clarifying the Dynamics of the Model 

{This task will also take place as a full group session} 

What is the dynamic relationship between each component and each other component, and 
between each component and the whole system in terms of three basic relational categories: 
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9:00 - 10:30 A. What does each component create (provide, stimulate, enhance, etc.) 

relative to each of the other component s and relative to the whole 
system? 

10:30 - 10:45 BREAK 

10:45 - 12:00 B. What does each component limit (define, delineate, demand, etc.) relative 

to each other component and to the whole system? 

12:00- 1:00 LUNCH 

1:00 - 2:30 C. What does each component sustain (ensure, reinforce, represent, etc.) 

relative to each of the other components and to the whole system? 

2:30 - 2:45 BREAK 



2:45 - 4:00 Task Three: Exploring the Roles and Responsibilities of Stakeholder Groups 

in the Implementation of a Comprehensive Accountability System 
{This activity will take place in three small groups each representing a particular 
stakeholder level— national, state, local — and will begin the identification of the 
ongoing tasks to be performed and the resources required for each level.} 

4:00 - 6:00 Plenary Reporting Session: following reports from each of the three working 

groups to the group as a whole, participants will reflect on how each stakeholder 
level can specifically contribute to the work of the other two levels. 

7:00 DINNER 
SUNDAY 

8:00 - 8:30 BREAKFAST 

8:30 - 9:00 Review of and reflection on the products developed r 

9:00 - 10:30 Task Four: Identification of Next Steps 

{The whole group will brainstorm key questions that need to be resolved and then the three 
small groups will develop specific recommendations) 

10:30 - 1 1 .00 BREAK AND HOTEL CHECKOUT 

1 1:00 - 1 1:45 Reporting, discussion and assignments for next steps 

11:45 - 12:00 Closing and Send Out 
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BRIEFING PAPER: ACCOUNTABILITY FOCUS GROUP 
INTRODUCTION: 

Leading and Managing for Performance. (NASDSE, 1993) supports educational restructuring and 
reform and calls for a greater focus on student performance and results. Accountability for 
implementation of the fundamental processes of FAPE is a vital component of this focus and 
NASDSE urges that the substantative test (educational benefit) for FAPE should become 
moreoperational. There appears to be an emerging consensus that a broad-based accountability 
"model" is needed to effectively assess the status of the education of students with disabilities. 
Components of a comprehensive accountability model already exist or are evolving. These 
components include compliance monitoring (input/process accountability), state assessment systems 
based on state determined standards (outcome accountability), and individual student accountability 
(accountability for individual student achievement and procedural protections). 

To initiate discussion of this topic, the NASDSE Board of Directors has scheduled a focus group 
meeting to convene February 24 - 26, 1995 in Salt Lake City, Utah. 

PURPOSE OF THE FOCUS GROUP: 

The overall purpose of this focus group is to begin to develop a shared vision of accountability for 
the education of students with disabilities. The development of a shared vision shall consider existing 
components, means of articulation across components, and potential roles that the U.S. Department 
of Education, state education agencies, local school systems and families have in the implementation 
of a comprehensive model of accountability. 

ANTICIPATED OUTCOMES: 

1. A shared understanding of the differences among input/process accountability, results-based 
accountability, and the accountability for individual student performance and due process. 

2. A shared understanding of the need for a multi-component and multi-level articulated model 
of accountability. 

r 

3. A shared understanding of the need for multiple stakeholder roles and responsibilities in the 
implementation of a comprehensive articulated model of accountability. 

4. A shared understanding of the relationship between an accountability system for the 
education of students with disabilities and the broad-based systems of accountability that 
undergird states' educational reform efforts. 

5. Initial ideas about how best to report nationally the status of the education of students with 
disabilities so that the creation of a unified system of accountability is prompted. 

6. A plan for sharing and refining the initial understandings and vision that are developed as a 
result of this meeting with other stakeholders. 
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Notes from Tom Hehir's Presentation at Monitoring Conference - 1 1/4/94 

@ 

As a part of the "reinventing government" activities that have been going on throughout the 
Executive Branch, OSERS has taken time to consider "Who are our customers?" It 
has decided that OSERS' customers are people with disabilities and sometimes 
their parents." 

0 

Those who are paid to deliver services to OSERS customers are OSERS' partners. 
OSEP is an "enforcer" and this must be a part of the partnership relationship. It is the 
same type of relationship that SEAs have with their LEAs. 

® OSERS enforces and implements. Although OSEP intends to put an emphasis on 

implementing, it cannot deny its responsibility to enforce. 

The biggest thing that OSEP's customers value is the monitoring and enforcement of IDEA - 
this is what the customers are saying. They think it a good law and they want it 
® protected. Although we might expect that everyone will do what IDEA requires just 

because it is right, some do not. The IDEA was passed because the 14 th amendment 
to the Constitution was not working for students with disabilities and there was 
widespread exclusion of students with disabilities from school. 

® Do we need to continue to monitor? Yes, because the education of students with disabilities 

requires school systems to do things differently from what they would otherwise do 
and it is unlikely that what is needed would be done without outside pressure. 
Therefore, monitoring is critically important - actually the most important thing done 

m . at OSEP. 

OSEP is now looking at a 3 year change effort to emphasize those aspects of monitoring that 
are most connected to results for students with disabilities. We must be conservative 
and not deny that inputs do matter while we add the emphasis on results. 

® The Longitudinal Study shows that students with disabilities in regular classes frequently get 

no supports or modifications and fail as a result; those who got the necessary support 
succeeded. This is b.n input issue that is critically important to success for students 
with disabilities. 

Q 

We have to avoid the naive romanticism of looking only at outcomes. Some states have 
dismantled their capacity to monitor compliance and this is misguided. 

Every instance of non-compliance is not equal. OSEP is trying to get away from this 
•j approach. We need to be able to focus on the things that are most important. For 

example, protecting rights and seeing that students with disabilities get FAPE is the 
big issue. OSEP monitoring reports will begin to reflect this emphasis. Easy issues 
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have had too much attention and the more difficult ones have not had adequate 
attention. 

Enormous variability exists from state to state in implementing LRE, a major influence on 
student results. We have to move students with disabilities away from inappropriate 
segregation. A state's educational finance system is an important variable that 
influences LRE. This is another complicated problem. Personnel issues are another 
enormous factor influencing LRE and quality. 

We have to look at systemic issues to make the system serve students better. The important 
point is the intersection of the input orientation and the results orientation. 

There are many implications for the federal-state relationship. The feds must use monitoring 
to identify the issues that states should be handling to move systems forward in IDEA 
implementation. We must move to what the law intended in the first place and 
monitoring is a powerful force in this effort. 

We have accomplished a lot in access for students with disabilities in the past 20 years, but 
we now have to go beyond that. Monitoring has had an important role in this 
achievement and it will continue to have an important role in the future. 

We have to take a developmental approach - study and phase-in changes to meet newly 
identified needs. 

It will be important to avoid an esoteric debate on "monitoring for quality." For example, 
monitors cannot judge the efficacy of a specific assessment for a student. We need 
to look at the total program a student is experiencing. We must also re-think the 
concept of the IEP. Quality can be inferred from results such as dropout rates, access 
to vocational training, etc. 

OSEP cannot overstep the bounds of legal authority. However, there is enough authority 
to do what has to be done to incorporate a results orientation. 

The administration is trying to coordinate federal educational policy and programs better than 
it has in the past. Reauthorization of IDEA will align more closely to Goals 2000, 
School-To-Work, and other federal education laws. However, it is clear that these 
laws all have different purposes. Such coordination is complicated and it is clear that 
it will be a while before meaningful coordination happens at the federal level. 
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A NEW PARADIGM— HIGH LEVELS OF SUCCESS FOR ALL 



"If schools are to be responsive to the different needs and talents of diverse learners, 
they must be organized to allow for variability rather than to assume uniformity. . . 
Rather than seek to make the current system of schooling perform more efficiently by 
standardizing practice, school reform efforts must focus on building the capacity of 
schools and teachers to undertake tasks they have never been called upon to accomplish. 
Schools and teachers must work to ensure that all students learn to think critically, to 
invent, to produce, and to solve problems. Because this goal requires responding to 
students ' nonstandardized needs, it far exceeds what teacher-proof curricula or 
administrator-proof management processes could ever accomplish. " 

Linda Darling-Hammond, 1993 
The Economic Mandate for School Change 

Across the nation, reform efforts are focussing on what students need to know and be able to do 
in a highly complex, technological society and global marketplace. Within this context of 
reform, educators, business leaders, policy makers, and legislators agree that the level of student 
achievement in American schools must be raised. The two parts to this national "call to arms" 
are: 1) too many students are failing to achieve even basic competencies; and 2) the world we 
live in requires skills that are not being explicitly addressed in schools. These two concerns have 
led to a nationwide emphasis on defining desired student outcomes, establishing standards of 
performance, and holding schools accountable for helping all students learn at high levels. 

The momentum to create high academic standards for all students has been fueled by the shift 
from an industrial-based economy to an information-based economy, a a that has left many 
American workers without the necessary skills to succeed. Because the demands of society are 
different today, the economy no longer has a place for individuals who are willing to work hard 
but have minimal skills. According to a report published by the Hudson Institute, more than half 
of the new jobs created in this decade will require education beyond high school, and 90 percent 
will require at least a high school education (Hudson Institute, 1987). 
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Numerous studies and reports have described the deficiencies and "skills gap" in the American 
workforce, concluding that unless educational performance in the United States improves 
dramatically, American workers will be unable to use the new technologies that will create most 
of the world's jobs and economic growth in the next century (United Way of America, 1 990, 
U.S. Department of Labor, 1991, Hudson Institute, 1987). The high school completion rate 
hovels around 75%, students with a diploma are not necessarily literate, and many of those who 
enter college or the workforce are not prepared to meet the increasing demands of a rapidly 
changing, technology-driven, and increasingly competitive market place. 
The need to link educational reform to a mission of ensuring that all students are prepared to 
meet the challenges of a high technology information age has been underscored by virtually all 
sectors, including corporate America, and improving student performance is the focal point of 
major policy initiatives nationally and in almost every state capitol. Such prominent groups as 
the National Business Roundtable, the National Governor's Association, and the Education 
Commission of the States have endorsed the concept of redesigning education around high 
standards of student performance as the fundamental principle of school reform. 
The emerging reform agenda is calling for a "new paradigm" of education that puts student 
achievement at the center of the school's mission, and holds schools accountable for providing 
educational opportunities that enable all students to acquire the necessary knowledge, skills, and 
personal orientations needed to succeed in adult life. An emphasis on higher levels of learning 
for all students means that schools must shift to a new paradigm driven by assumptions, 
principles, and practices that contrast sharply with the current paradigm on which schooling in 
America is based. 

For the past 100 years, our schools met the workforce needs of an industrial society by 
organizing learning around a standardized curriculum delivered in standardized time periods 
called Carnegie Units. Within this structure, curriculum is defined as a set of units, sequences, 
and facts, and grades are based on the averaging of performance over a fixed period of time. 
Credentials (Carnegie units) are awarded based on "time served", and the failure of significant 
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numbers of students is not only accepted, but regarded as an expected result of norm-referenced 
testing. The technology-driven information age requires a very different approach to education. 
Today's workplace already requires individuals to understand multidimensional problems, 
design solutions, plan their own tasks, evaluate results, and work cooperatively with others. 
These changes represent a new mission for education that requires schools not merely to deliver 
instruction but to be accountable for ensuring that educational opportunities result in all students 
learning at high levels. The teacher's job is no longer to "cover" a time-based curriculum, "but 
to enable diverse learners to construct their own knowledge and to develop their talents in 
effective and powerful ways." (Darling-Hammond, 1993). 

The Mandate for Equity 

The nationwide focus on student performance has heightened concerns about the educational 
outcomes of students with diverse needs, including students with disabilities. Echoing other 
educational constituents seeking answers about the outcomes that result from special education is 
a report of the National Council on Disability (1989), which summarized the views of parents, 
educators, taxpayers, and others. 

"The time has come to ask the same questions for students with disabilities that we have 
been asking about students without disabilities: Are they achieving? Are they staying in 
school? Are they prepared to enter the work force when they finish school? Are they 
going on to participate in post-secondary education and training? Are they prepared for 
adult life?" 

In its Twelfth and Thirteenth Annual Reports to Congress on the Implementation of The 
Education of the Handicapped Act (now entitled the Individuals with Disabilities Education Act), 
the Department of Education reported dismal outcomes for students with disabilities. In 1987- 
88, nearly one-third received a failing grade in at least one of their courses, and in 1988-89, only 
slightly more than one half of students with disabilities exiting the school system left with a 
diploma or certificate. Despite the similarities between students who are labelled as learning 
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disabled and those who are not, in 1988-89 more than 80% of the learning disabled students 
received educational services outside the regular classroom. 

The National Longitudinal Transition Study (NLTS) collected school performance data on more 
than 8,000 youth between the ages of 13 and 23 who were in special education programs in the 
1985-86 school year. The study determined that 1 in 10 students did not receive grades in my 
courses, and that a third of the students considered to be in graded programs received a failing 
grade in one or more courses. About a third of the 8,000 students dropped out of school over the 
two year period; and the dropout rate for students with a learning disability was 36% (Wagner, 
1989). 

Underlying the momentum to create standards, better schools, and better educated students is a 
strong belief that "students who have been traditionally allowed to fail must be helped to 
succeed, and that many more must become not just minimally schooled but highly proficient and 
inventive (Darling-Hammond & Snyder, 1993). More than ever before in our nation's history, 
policymakers and the public are recognizing that educational failure and undeveloped human 
talent are permanent drains on society. 



"To fulfill the old promise of American education— that students will be prepared to take 
their place in society-requires a new level of performance for the system, and a new 
level of effort at reform. . . Standards-based reform seeks to establish clear attainable 
standards at internationally competitive levels: for the entire student population. This 
represents a new way of thinking— -a paradigm shift— about American students. It raises 
our expectations for every student in every school, not just some students in some 
schools. " 

National Education Goals Panel, 1993 
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The New Paradigm — A Catalyst for School Reform 

The current movement to reform American schools represents a large-scale shift in national and 
state-level policy making toward performance-based reform (Finn, 1990). The paradigm that 
puts learner outcomes at the center of the school's mission has shifted the purpose and direction 
of reform initiatives to an emphasis on higher levels of learning for all, and school accountability 
to key stakeholders such as parents, business representatives, and community members. A 
conceptual transformation is occurring in education today, and improving the level of student 
learning in America's schools has become the focus of educational policy thinking and action 
across the nation. State policy making bodies have placed the debate over what students should 
know and be able to do at the center of major statewide efforts to define standards of 
achievement, design new curricular frameworks, and adopt richer assessment systems to measure 
students' and school performance (Wilhoit,1992). This shift has fundamentally altered 
traditional views of school improvement. 

Many educational change experts recognized the need for this shift a decade ago. In 1983, 
Theodore Sizer, the founder of the Coalition of Essential Schools, described it as "one that 
rejects time-based, means-based, bell-curve-based schooling in favor of results-based, success- 
based schools" (Sizer, 1992). Dr. Philip Schlecty of the Center for Leadership on School Reform 
describes the "mandate for change" this way: 



"Altering the rules, roles, and relationships that govern the way time, people, space, and 
technology are deployed and used so that schools are organized around children and the 
work we want them to do, and so that communities are organized to support the creation 
of conditions that will allow those children to succeed at what we want them to do. " 

Schlecty, 1993 



Throughout the history of educational change, the emphasis has been on improving the process 
of education — changing instructional methods and practices, adopting new programs, altering 
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schedules, etc. Educators and reformers focussed on the content of education — on changing the 
curriculum, the textbooks, the lessons and adding new programs. Hundreds of learning 
objectives (which were thought of as outcomes) were derived from existing curricula (content) 
rather than designing curricula to facilitate the achievement of intended outcomes representing 
what students should actually know and be able to do. Our attention to the product of 
education — the educational achievement of students — was largely ignored in the old paradigm. 
New programs, practices, and curricula may have been 'Validated" by virtue of having produced 
desirable results where they were developed and evaluated, but when these innovations were 
adopted by others, their impact on students was rarely assessed, and often non-existent. 

Efforts to reform schools over the past several decades conformed to the industrial model, 
regarding schools as bureaucracies with various layers of management and instructional delivery 
that could be improved by increasing efficiency. The reforms of the 1970's tried to standardize 
the quality of schooling by mandating curriculum guides and narrowing textbook selection. 
When the effective schools movement began, it focussed on the common characteristics of "high 
achieving" schools. These characteristics, called "correlates" of school effectiveness, became the 
foundation of hundreds of school improvement initiatives across the country. The first and last 
of these correlates, as expressed by Edmunds, Lezotte, and others, were thought of as the 
"pillars" of school effectiveness — an emphasis on success for all students and frequent 
monitoring of student progress toward achieving success. However, while many schools 
involved in the effective schools movement made a commitment to frequent monitoring of 
student progress, progress was tracked in terms of very limited notions of educational 
success — getting satisfactory grades, getting promoted, and accumulating credit. Although the 
school improvement models of the '80s sought to define the various dimensions of school 
effectiveness, few of them directly addressed the outcomes of education. Achievement was 
defined in terms of the educational process, and educators became chagrined to learn that a 
surprising number of students who had achieved these educational milestones did not have the 
knowledge, skills, attitudes and personal qualities that are prerequisites to success in the "real 
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world" (Lezotte,1991, Lezotte and Jacoby, 1992, Eaker, Ranells, and DuFour, 1991, Levine and 
Lezotte, 1990). What we learned from the effective schools movement was that implementing 
new programs and practices does not ensure better results. 

"Unfortunately, site-based innovations mean nothing if a school cannot determine if the 
efforts have had an effect on students. Most schools move from innovation to innovation 
('We are doing whole language, or cooperative lear ning, or curriculum integration ') 
and define success as the implementation of the i^ist innovation. To be blunt, this is 
nonsense. What difference does any innovation make if a school cannot determine 
effects on kids? " 

Glickman, 1992 

In the past, special education also was driven by an emphasis on the content and process aspects 
of reform rather than a systematic examination of the outcomes and benefits that resulted for 
students. For more than a decade after the enactment of P.L. 94-142 in 1975, attention at 
national, state, and local levels focussed on compliance with the procedural provisions of the Act 
that were designed to ensure access and equity for students with disabilities. Today, however, 
special educators have also shifted to an emphasis on defining appropriate outcomes for students 
with disabilities, and on determining the extent to which various programmatic and instructional 
approaches result in students' acquiring the knowledge, skills, and orientations they need to lead 
productive and fulfilling lives. 

More and more educational leaders are recognizing that embarking on school reform without 
defining and mojutsmng the intended impact of reform initiatives on educational results does not 
lead to improved outcomes or higher levels of learning for students. The emphasis on student 
outcomes and school accountability that is driving current national and state-level reform efforts 
thus represents an important shift in our orientation toward educational change and innovation in 
America's schools. 
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THE MOVEMENT TO DEFINE STANDARDS AND OUTCOME FRAMEWORKS 

"In the absence of national standards, we have evolved a haphazard, accidental, 
disconnected national curriculum based on mass-market textbooks and standardized, 
multiple-choice tests. Education reform must begin with broad agreement on what 
children should learn. Learning, after all, is the heart and soul of education. When 
there is no agreement regarding what students should learn, then each part of the 
education system pursues different, sometimes contradictory goals. " 

Diane Ravitch, 1993 

National Goals and Standards 

Under the Goals 2000: Educate America Act, the National Education Goals adopted by the 
nation's governors in 1990 have been codified into law. The National Governor's Association 

proposed the national goals as a vision of change for all children. The goals were designed from 

a belief that "efforts to restructure education must work toward guaranteeing that all students are 

engaged in rigorous programs of instruction designed to ensure that every child, regardless of 

background or disability, acquires the knowledge and skills necessary to succeed in a changing 

economy" (National Governor's Association, 1990). 

Secretary of Education, Richard W. Riley described the National Education Goals as a shared 
vision "of the education system our country needs for the 21st century. It is one in which schools 
help every child (regardless of her background or where he attends school) to reach challenging 
academic standards and leave school prepared for responsible citizenship and a productive future. 
. . It is of an education system that is committed to producing real results, for all of its studer's" 
( U.S. Department of Education, Community Update, September, 1993). Under Goals 2000, the 
National Goals have set the following vision for America's schools. 
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By the year 2000: 

1) All children will start school ready to learn; 

2) The high school graduation rate will be at least 90%; 

3) All students will leave grades 4, 8, and 12 competent for their level in English, math, 
science, foreign languages, civics and government, economics, arts, history and 
geography, and the capacity to use their minds well for responsible citizenship, further 
learning, and productive employment in our nation's modern economy; 

4) The nation 's teaching force will have access to professional inservice training programs; 

5 ) United States ' students will be first in the world in math and science achievement; 

6) Every adult American will be literate and will possess the knowledge and skills necessary 
to compete in a global economy and exercise the rights and responsibilities of 
citizenship; 

7) Every school will be free of drugs, violence, and the unauthorized presence of firearms 
and alcohol and will offer a disciplined environment conducive to learning; and 

8) Every school will promote partnerships that will increase parental involvement and 
participation in promoting the social, emotional, and academic growth of children. 

By focussing on improving the knowledge and skills of the nation's students, the National 
Education Goals provide a foundation for developing national standards that are intended to: 1) 
identify what students need to know and be able to do to live and work in the 21st century; 2) 
raise the achievement of all students; 3) ensure that all students have equal educational 
opportunity; and 4) create a coherent and consistent approach to education in the nation's 
educational system. The standards movement reflects widespread recognition that the lack of 
consensus about what students should leam in the nation's schools is contributing to the erosion 
of student achievement. Proponents of standards ;rgue that "when no one agrees on what 
students should learn, then each part of the educational system pursues different, and sometimes 
contradictory, goals. As a result, the education system as a whole is riddled with inequity, 
incoherence, and inefficiency" (U.S. Department of Education, 1992). 
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The work of the National Council of Teachers of Mathematics (NCTM) demonstrated the power 
of national curriculum standards to drive education reform in a coherent manner. Published in 
1989, the NCTM Curriculum and Evaluation Standards for Mathematics were created through a 
review and consensus process involving a wide array of participants including business leaders, 
parents, mathematicians, and thousands of teachers who collaborated on what students should 
know and be able to do in mathematics. The NCTM used a similar consensus process to develop 
the Professional Standards for Teaching Mathematics and recently began a two-year effort to 
develop standards for math assessment that will expand upon concepts contained in the 
curriculum-standards document. 

What is unique about the NCTM effort is that it evolved from the initiative of the professional 
community. The momentum and authority to develop the mathematics standards didn't come 
from a government agency or corporate group, but from the nation's math teachers who had 
surveyed their field and decided it needed to change. The NCTM Standards represent an 
empowering vision of school mathematics for all students. They are based on the premise that 
all students need to learn more and different types of mathematics in order to be productive 
citizens in the 21st century. The Standards emphasize that "students need to learn to value 
mathematics, to reason and communicate mathematically, and to become confident in their 
power to use mathematics coherently to make sense of problematic situations in the world around 
them" (Romberg, 1993). Since their publication, the NCTM Standards have had a significant 
impact on curriculum development efforts, instructional practices, assessment, staff 
development, and teacher education at every level. Every commercial mathematics textbook 
published since 1989 has claimed to incorporate the NCTM standards, and according to the 
NCTM, by 1992 about a third of the nation's mathematics teachers were using the new standards 
(Ravitch, 1993). A survey by the Council of Chief State School Officers indicated that at least 
41 states have realigned or are in the process of realigning their state frameworks with the 
NCTM Standards (Blank and Dalkilic, 1992). 
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Following the work of NCTM, in 1992, the National Council on Educational Standards and 
Testing (NCEST), a bi-partisan group established by Congress, published the report, Raising 
Standards in American Education which put the issues of standards and assessments before the 
public. NCEST recommended the development of high standards that were national, not federal, 
(i.e., not controlled by the federal government), voluntary, not mandatory, and dynamic, not 
static. Their report emphasized that the standards should be developed through a participatory 
process and should be used to provide focus and direction to curriculum reform in various 
content areas, but not to create a national curriculum. NCEST did not recommend a single 
national examination, but rather "a national testing system that was fair and equitable, in which 
different tests would be linked to common standards" (Ravitch, 1993). NCEST provided 
clarification of the meaning of standards by recommending that the national standards be 
developed to include: 1) conte nt standards— what students should know and be able to do; and 
2) performance standards— the level(s) of student competence in the content (O'Neil, J., 1993, 
Selden, R., 1992). 

Proponents of national standards and assessments consider them to be the "best way to propel 
education reform forward, from the state policy level to classroom instruction," and the most 
effective means of quickly energizing the entire system (National Association of State Boards of 
Education, 1992). The Goals 2000: Educate America Act is intended to provide a framework 
for meeting the national goals by promoting coherent nationwide, systemic education reform to 
"ensure equitable educational opportunities and high levels of achievement for all American 
students," providing a frarnework for reauthorization of all Federal education programs, and by 
promoting the development and adoption of "a voluntary national system of skill standards and 
certifications." The law establishes the National Education Goals Panel and the National 
Education Standards and Assessment Council which will develop and certify voluntary national 
content, performance, and opportunity-to-learn standards, as shown in 
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FIGURE 1 



National Standards Model from the NCEST Report 



STANDARDS 
FOR 

STUDENTS 



SCHOOL 
STANDARDS 



Content Standards: Define what children should know 
and be able to do. They describe the knowledge, skills, and 
understandings students should have in particular subject 
areas in order to attain high levels of proficiency. They 
provide guidelines for what schools should teach to ensure 
that all students are prepared to live and work in the 21st 
century 



Performance Standards: Identify the levels of 
achievement in the subject matter set out in the content 
standards. They define how good is good enough by setting 
specific expectations for student performance and various 
levels of proficiency. 



Opportunity-to-Learn Standards: Refer to the 
conditions in schools that enable all students to have a fair 
opportunity to achieve the knowledge, skills, and 
understandings set out in the content standards. They 
address such areas as curriculum, instruction, assessment, 
technology and other resources, a safe environment, and 
professional development. 



Figure 1 above. The legislation does not propose a single national examination. The legislation 
calls for states to submit a Strategic Educational Reform Plan developed by a multi -constituency 
group. If the plan is approved, the state will qualify for funds to implement the plan, including 
funds that can be awarded to schools that undertake reforms called for in the legislation. 
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Currently, at least 1 1 national professional and subject-matter groups are spearheading efforts to 
set academic standards for their own disciplines, and the U.S. Department of Education has 
funded national standards projects in the arts, civics, English, foreign languages, geography, 
history, and science. For the first time, specialists in various disciplines are setting aside their 
parochial interests to agree on a core set of knowledge and skills that all students, not just high 
achievers, should be taught. However, the education standards resulting from these efforts will 
be anything but standardized. The projects are highly diverse in terms of funding levels (ranging 
from $30,000 to over $3 million), timelines, scope of participation, and specificity of learner 
outcomes. Some projects are developing detailed guidelines, while others are defining broad 
themes on which to base instruction. The boards that oversee some of the standards projects 
have a high representation of scholars, while others rely more on teachers. As the standards- 
setting movement gains momentum, concerns are being raised that the documents may be too 
numerous, lengthy, and different from one another, and contain too much to teach (Viadero and 
West, 1993, Viadero, 1993). Thus, while all of the initiatives appear to be significantly 
influenced by the work of the NCTM, educators will be seeing more diversity than uniformity in 
the approaches and the quality of standards produced. 

Outcome Frameworks 

Paralleling the interest in national standards has been the emergence of outcome frameworks 
intended to provide direction in determining what students should be taught to meet both the 
challenges and the promises of adulthood in an increasingly technological and changing world. 

The SCANS Renorts 

Notable at the national level is the outcome framework developed by the Secretary of Labor's 
Commission on Achieving Necessary Skills (SCANS), which focussed on how schools should 
prepare young people for work. The Commission's charge was to define the skills needed for 
employment today in all manner of jobs. The first SCANS report, What Work Requires of 
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Schools: A SCANS Report for America 2000, emphasized that the demands on business and 
workers are very different with the "globalization of commerce and industry and the explosive 
growth of technology on the job." The report concluded that "all American high school students 
must develop a new set of competencies and foundation skills if they are to enjoy a productive, 
full, and satisfying life", and that the nation's schools must be transformed into high- 
performance organizations "relentlessly committed to producing skilled graduates as the norm, 
not the exception" (U.S. Department of Labor, 1991). 

Five competencies were identified, which, in conjunction with a three-part foundation of skills 
and personal qualities "lie at the heart of job performance today", and represent essential 
preparation for all students. The report emphasized ". . .seldom does one of these eight 
components stand alone in job performance. They are highly integrated, and most tasks require 
workers to draw on several of them simultaneously." The five competencies and three 
foundation skills, referred to in the SCANS document as "workplace know-how", are depicted in 
Figure 2. 

A second report, Skills and Tasks for Jobs: A SCANS Report for America 2000, included 
hundreds of detailed, job-related examples of how the competencies and foundation skills would 
be used (and required) in various job tasks. A third report, Learning a Living: A Blueprint for 
High Performance, provides specific examples and recommendations for integrating the SCANS 
competencies into the curriculum, and compares traditional approaches to various skills (e.g., 
writing) to approaches that are consistent with workplace requirements. 

The SCANS Reports begin from the premise that while "schools do more than prepare young 
people for work", all of the major national reform efforts recognize a need "to link education to 
the real world. All seek a particular kind of learner, one who can put knowledge and skills 
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FIGURE 2 

Workplace Know-How 

The know-how identified by SCANS is made up of five competencies and a three-part foundation of 
skills and personal qualities that are needed for solid job performance. These are: 

WORKPLACE COMPETENCIES— Effective workers can productively use: 

• Resources — They know how to allocate time, money, materials, space, and staff. 

• Interpersonal skills — They can work on teams, teach others, serve customers, lead, 
negotiate, and work with people from culturally diverse backgrounds. 

• Information — They can acquire and evaluate data, organize and maintain files, interpret 
and communicate, and use computers to process information. 

• Systems — They understand social, organizational, and technological systems; they can 
monitor and correct performance; and they can design or improve systems. 

• Technology — They can select equipment and tools, spply technology to specific tasks, 
and maintain and troubleshoot equipment. 

FOUNDATION SKILLS— Competent workers in the high-performance workplace need: 

• Basic Skills— reading, writing, arithmetic and mathematics, speaking and listening. 

• Thinking Skills— the ability to learn, to reason, to think creatively, to make decisions, 
and to solve problems. 

• Personal Qualities — individual responsibility, self-esteem and self-management, 
sociability, and integrity. 

SCANS, 1992 

into practice as a productive worker, a responsible citizen, and a morj complex human being." 
To the extent that the SCANS Reports deal with issues of equity and diversity, they do so with 
respect to minority and low-income workers and those with limited proficiency in English. 
While they do not directly mention students or workers with disabilities, they do argue forcefully 
for respecting differences in people: 
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"Education in the SCANS skills must begin with the realization that there are many 
paths to the same goal; that assessments should play to students ' strengths, not their 
weaknesses; and that tests should not needlessly penalize students who need more 
time. . . Variation and diversity are not the enemies of high-quality education. The 
enemy is rigid insistence on a factory model of schooling, a prescription for failure 
that refuses to accommodate diversity or to allow those students with special 
strengths to function productively. " 

Learning a Living: A Blueprint for High Performance, 1992 

The SCANS reports continue to influence the development of outcome and curriculum 
frameworks at the national, state, and local levels, because they have brought core competencies 
and foundation skills to life for policy makers, educators, and students alike. Moreover, the 
SCANS framework of competencies and skills are seen as highly relevant to life-roles other than 
work. 

An Outcome Framework for Special Education 

The National Center on Educational Outcomes (NCEO) at the University of Minnesota, was 
funded by the Office of Special Education Programs to develop an outcome framework and a set 
of outcome indicators for students with disabilities. NCEO's Conceptual Model of Educational 
Outcomes was developed in collaboration with the National Association of State Directors of 
Special Education (NASDSE) through a consensus process involving hundreds of educators, 
policy makers, administrators, and parents. It is viewed by NCEO as an inclusive framework 
that is applicable to all students, not just students with disabilities. Shown in Figure 3, the model 
includes eight outcome domains. Two of the domains, Presence and Participation, and 
Accommodation and Adaptation are placed within the context of the educational process 
itself — i.e., NCEO proposes that an examination of outcomes for all students, including students 
with disabilities, must take into account their level of participation in school and their adaptation 
and coping skills. The six other domains included in the model are Physical Health, 
Responsibility and Independence, Contribution and Citizenship, Academic and Functional 
Literacy, Personal and Social Adjustment, and Satisfaction. 
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For each of the domains, the model identifies outcomes, indicators of outcomes, and data sources 
for the various indicators. The domain of Academic and Functional Literacy includes five 
overall outcomes, two of which reflect the nationwide emphasis on higher order thinking skills 
and use of technology. The five outcomes for which students must demonstrate competence are: 
communication; problem-solving strategies and critical-thinking skills; math, reading, and 
writing skills; other academic and nonacademic skills; and using technology. NCEO views its 
model as providing a framework and examples that can be used by states, districts, and schools. 
NCEO is also using a consensus-building process to identify outcomes and indicators for 
developmental levels that will span from 3 years of age to post-school age. 

State Frameworks 

Over the past several years, many states have started to create curriculum frameworks that 
represent a new vision of what students should learn in school and a more integrated approach to 
education. According to the National Association of State Boards of Education, at least 30 states 
have identified essential student outcomes. While various states differ philosophically about 
what is important to include in the school curriculum, the underlying purpose of most of the 
states' efforts is to align curriculum and instruction with clear definitions of student learning. 

The state initiatives are often linked to state assessment programs or curriculum frameworks, and 
attempt to provide direction for reform at the local level. California was the first state to develop 
and use a state curriculum framework as a master plan for reforming instruction, assessment, 
textbooks, staff development, and teacher education. 

Some states (Connecticut, Maine, Michigan, and South Carolina) have developed frameworks 
that tie learner outcomes to a thematic organization of subject matter. These frameworks have 
been designed to provide a vision of the opportunities to learn that schools should provide for 
students. Other states, including Florida, Kentucky, Pennsylvania, and Vermont, 

Page 17 



Learner-Based accountability 



FIGURE 3 
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are seeking to integrate "real-life" workforce characteristics into the curriculum, "attempting to 
ensure that students perform at significantly high levels and meet expectations held by schools 
and future employers" (Olsen and Massanari, 1992). 



Maine and Virginia developed frameworks called a "common core of learning" to articulate a 
vision of education in their respective states. Through the notion of a "learning core," these 
states did not attempt to define everything that students would learn during their education 
experience, but rather what must be common to all students. Maine's Common Core of Learning 
(1990) was intended to challenge traditional beliefs anout students and schooling by providing a 
conceptual model of what students will need to know in the 21st century in the following four 
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areas: Personal and Global Stewardship; Communication; Reasoning and Problem-Solving; and 
the Human Record. Organized as a cross-disciplinary integrated approach to teaching and 
learning, it was developed as a basis for charting the course of educational change in Maine. 

Regarding the participation of students with disabilities, the state-sponsored commission of 
business and educational leaders, professionals, and parents that developed the Maine framework 
stated, "we believe exceptional students should experience the Common Core of Learning to the 
degree that they are able, with the support of teachers who build on their strengths. The 
Individual Education Plan (IEP) is the vehicle for identifying which learning outcomes are 
appropriate for a given student." 

Virginia developed a program called World Class Education with a Common Core of Learning 
as its centerpiece. Six principles guide the Virginia program: 1) a focus on outcomes and the 
results of education; 2) schools will be held accountable for their results; 3) the assumption that 
all students can learn; 4) an emphasis on collaboration, rather than competition; 5) an emphasis 
on active, constructed, and connected learning, drawn from a variety of content areas and related 
to real problems; and 6) the role of assessment in supporting better teaching and learning 
(Bradford and Stiff, 1993). Virginia's Common Core of Learning was not designed as a state 
curriculum, but rather as an outcomes-based framework for voluntary school improvement. 

What Standards and Outcome Frameworks Mean 
For Students With Diverse Needs 

Understandably, those who advocate education reform on behalf of students with disabilities 
have expressed concerns about the implementation of the national goals and have identified 
major issues about how the achievement of students with disabilities will be assessed in meeting 
the goals. Advocates have emphasized that students' needs for special accommodations to 
complete tests must be taken into consideration, and have pointed to research showing that many 
schools leave student;; with disabilities out of testing programs to boost overall district scores 
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(DeStefano, L. and Nietzer, D., 1991; Schrag, J., 1991). Each year, the National Education 
Goals Panel produces a report on the nation's progress toward the national goals. However, most 
of the sources of information used for the report do not include students with disabilities. The 
National Center on Education Outcomes (NCEO) pointed out that in many cases, the lack of 
progress information for students with disabilities on goals that focus on academic achievement 
and performance is related to the exclusion of students with disabilities from national data 
collection programs (McGrew, K.S., Thurlow, M.L., Shriner, J.G., and Spiegel, A.N., 1992). 

The NCEST report did not address the issue of how students with disabilities would be 
accommodated in a national standards and testing system, and concerns have been raised about 
how a national system will deal with the issue of equity for these students and for poor and 
minority students. Several national groups are calling for broader collaboration and consensus 
about how to make high standards work for all students. These groups have emphasized that 
"democratic standards-setting must keep equity central. We cannot develop a list of new 
standards and then ignore the savage inequalities in the opportunities students have to learn" 
(Stewart, 1993). The following statement is an excerpt from a document entitled Criteria for 
National Testing Proposals which is signed by more than 50 education and civil rights leaders, 
including James Comer, Linda Darling-Hammon, Keith Geiger, John Goodlad, and Ted Sizer. 

". . .we believe that any real effort to create accountability in American schools must focus 
equal or more attention on improving the capabilities of children to learn and schools to teach 
as it does on gauging educational 'outcomes '. . . Given the tremendous differences between 
today 's achievements and the goals set for America 2000, the inadequate supports for children 
and families in American society, and the dramatic inequalities among schools ' resources, any 
policy to establish benchmarks for achievement without creating equity in the educational 
resources available to children would be a cruel hoax. " 

National Association of State Boards of Education, 1992 
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Supporters of national standards struggle with some basic issues that have important implications 
for special education. One basic question focusses on whether it is possible to develop a relevant 
set of content standards for all students, including those with special needs. O'Neil points out 
that "all of the efforts to establish national standards in various subject areas at this point affirm 
the goal that all students should be expected to master a core set of eonieni standards." However, 
opinions diverge on the matter of student performance standards, i.e., levels of student attainment 
of the content standards. On one side of the discussion are the proponents of one set of 
performance standards and a mechanism for measuring and reporting performance for all 
students on these uniform standards. This view assumes that multiple performance standards 
would have negative results by encouraging differential expectations and tracking. Others point 
to the need for differentiated standards and emphasize the importance of recognizing that some 
students will develop more specialized expertise in certain content areas. This view also reflects 
the belief that outcomes for students with disabilities should reflect the individual and diverse 
educational needs of those students. It assumes that performance standards for some students 
with disabilities will be differentiated at certain points in the curriculum or at certain age or grade 
levels. 

In its recent report to the National Education Goals Panel, Promises To Keep: Creating High 

Standards for American Students, the national Technical Planning Group for Goals 3 and 4 

(goals aimed at academic achievement) emphasized that subject area content standards must be 

Hpvplnppnp.ntallv appropriate: 

"The standards proposed should support and challenge students achieving at all 
performance levels. While they should not represent minimum expectations, the 
standards should be suitable to and within the capabilities of students to learn. 
Regardless of students ' perceived ability, the standards should be achievable with 
proper supports and sustained effort. They should build appropriately on students ' 
developed capabilities at the elementary, middle, and high school levels of schooling. 
Any student who works hard in a good program should be able to mee> >e standards, 
and any student who meets the standards should be well prepared for hi or her 
future. " 
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The Technical Planning Group emphasized that "all students should be held to high and ® 

appropriate standards, and should be included in efforts to characterize the nation's level of 

education achievement." Their report made specific reference to the importance of high 

standards for students with disabilities: @ 

"The purpose of standards-based reform is to include everyone in deeper 
understanding of the most important and enduring knowledge and skills. To succeed, 
the nation must raise achievement at all levels — among the most able as well as the 

average and the disabled. " @ 

The report indicated that the standards set by the national professional organizations will be 

appropriate for many students now served in special education, recognizing that "for students 

with some disabilities, it might be appropriate to modify the conditions of instruction and ® 

methods of assessing attainment of those standards." 

"The standards discussed in this report would apply directly to all students except 

those, like the severely mentally retarded, whose individual diagnosis implies a ® 
judgement that the student cannot meet them. The Technical Planning Group defers 
to health and special education professionals to identify on a case-by-case basis the 
standards, both the content and level of performance, appropriate for these students. " 

The challenge facing special educators is to achieve consensus on appropriate outcomes for 

students with disabilities. The fundamental question is whether to have the same outcomes for 

all students, or to differentiate outcomes for different levels of ability or functioning. The 

publication ISSUES & OPTIONS In Restructuring Schools and Special Education Programs © 

(McLaughlin and Warren, 1992) outlined some of the assumptions, strategies, and implications 

that relate to each option. Having a unified set of outcomes for all students assumes that there is 

one set of educational outcomes to which all students are entitled and which all students can 

attain. The risk is that the educational goals and needs of students with disabilities, particularly 

those with moderate or severe disabilities, may go unnoticed or not be reflected in the outcomes 

all students must achieve. A unified set of outcomes assumes that there is a unified curriculum 

based on a common core of knowledge that all students must have. 
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"Within a unified curricular framework, students with disabilities receive instruction 
in the broad curricular domains, but at levels commensurate with their current 
functioning and with instructional modifications, as needed. The primary need is for 
breadth and balance— meaning that the curriculum should be defined not in terms of 
narrow subject matter but broader areas of knowledge and skill. A unified curriculum 
accompanies the concept of a unified system and responds to a unified set of 
outcomes. " 

McLaughlin and Warren, 1992, p. 61 

Having a differentiated set of outcomes assumes that "some students with disabilities may have 
unique educational needs that require a separate set of outcomes and performance measures that 
can be used for accountability purposes, and that it is educationally acceptable for those students 
to have different outcomes" (McLaughlin and Warren, 1992, p. 47). This option requires 
curricula with distinct alternatives designed to meet the unique educational needs of certain 
students with disabilities. The risk is that this approach will result in further separation of these 
students from the regular curricula, and could result in increased referral and identification if 
regular education viewed the alternative outcomes as less stringent, "thus providing a safety 
valve for students who are failing in the regular system." 

Beneath the surface of the current dialogue around standards for students with special needs is a 
question of whether the American dream truly belongs to all students, and whether American 
society is morally committed to equal opportunity. Believing that high standards of learning are 
appropriate and even necessary for all students requires fundamental changes in our approach to 
schooling. 
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"In the past how students were taught was mostly fixed, and the results 
varied— some students failed, most learned at least some of what they were 
taught. To enable all students to learn at high levels, varied instructional 
strategies are needed to challenge them. The standards are fixed but the means 
of reaching them are varied. . . High standards for all is a way to say that we 
will refuse to settle for low levels of learning for any student. . . All students will 
have opportunities to learn at higher levels when American society acts on its 
belief that this result is important now and in the future, it is fair, and it is 
possible. " 

National Education Goals Panel, 1993 

Proponents of national standards hope they will provide the leverag e needed to address equity 
issues — the necessary leverage to overcome the low expectations set for so many students, to 
ensure that all schools and teachers are aiming at the same high goals, and to motivate states and 
local districts to provide the resources necessary to provide all students with equal opportunities 
to meet high standards. Standards help to clarify that the purpose of schooling is not to sort 
people into artificial and often limiting groups, but to make the knowledge and skills that are 
essential to success in today's society accessible to all so that all students have the chance to 
achieve their full potential. 

SEAs will have to provide leadership in building a consensus on appropriate outcomes for 
students with disabilities in their states. First of all, consensus needs to be built regarding the 
extent to which educational outcomes for students with disabilities should be the same or 
different from the outcomes defined for students in general education. States have taken very 
different positions on this issue. For example, Kentucky has emphasized the inclusion of ail 
students in general education outcomes that focus on the application of academic and problem- 
solving skills (Kentucky Council on School Performance, 1989). Michigan has taken the 
opposite approach by developing specific outcomes for each disability category (Frey, 1991). 
These outcomes are not intended to replace general education outcomes, but to define the skills 
that students with specific disabilities will need in order to achieve the outcomes defined for the 
general population. 
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Numerous national organizations and agencies have studied the issues surrounding the 
implications of outcomes and accountability measures for students with disabilities and related 
national initiatives. Their reports highlight the challenges of equity, inclusion, high standards, 
and parallel versus unified delivery systems. These groups include the Office of Special 
Education Programs (OSEP), U.S. Department of Education, the National Association of State 
Boards of Education, the National Association of State Directors of Education, the Council for 
Exceptional Children, the American Association of School Administrators, the National 
Association of Secondary School Principals, the National Association of Elementary Principals, 
the National Parent Network on Disability, and others. These groups have been active 
participants in a far-ieaching dialogue related to defining life-long outcomes that span the full 
range of abilities and needs that will ultimately impact on how educational programs are 
structured for students with disabilities in the future. 
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LEARNER-BASED ACCOUNTABILITY — A FOUNDATION FOR SCHOOL REFORM 

"Accountability is achieved only if a school 's policies and practices work both to 
provide an environment that is conducive to learner-centered practice and to identify 
and correct problems as they occur. . .accountable schools institute practices for 
feedback and assessment, safeguards to prevent students from 'falling through the 
cracks, ' and incentives to encourage all members of the school community to focus 
continually on the needs of students and the improvement of practice. " 

Linda Darling-Hammond & Jon Snyder, 1993 

New Expectations for Student Success 

A commitment to higher levels of learning for all students is a commitment to learner-based 
accountability, and this commitment is central to the emerging paradigm that is driving 
educational reform efforts. It is creating new roles for teachers and new perspectives of school 
accountability. Establishing this commitment is a major challenge for schools and requires an 
effort of sufficient intensity to overcome political and substantive barriers" (Darling-Hammond 
and Snyder, 1993). The school restructuring movement has become a major vehicle for helping 
schools shift to a new paradigm that reorganizes schools to be genuinely accountable for their 
students and to their communities. It makes student learning and school accountability the 
central elements of meaningful reform. Restructuring activities "change fundamental 
assumptions, practices, and relationships both within the organization and between the 
organization and the outside world in ways that are intended to result in higher levels of learning 
for all students. Unlike past reforms that often addressed elements of the educational system 
separately and focussed on improving the existing organization, restructuring addresses the 
changes needed in the total system and all of the interlocking influences on student performance" 
(Conley, 1993, David, 1991). 

The central questions that drive school restructuring efforts are: What do we want students to 
know and be able to do? What kinds of learning experiences produce these outcomes? What 
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does it take to transform schools into places where this happens? In their extensive efforts to 
document the policy and organizational requirements of school restructuring, Darling-Hammond 
and her colleagues at the National Center for Restructuring Education, Schools, and Teaching 
(NCREST) have described new concepts of accountability that emphasize high levels of learning 
for all rather than the "traditional school outcomes of success for some and failure for many 
others." They call these forms of accountability "learner-centered, since they seek to focus on 
the needs and interests of learners for appropriate and supportive forms of teaching, rather than 
on the demands of bureaucracies for standardized forms of schooling" (Darling-Hammond, 
Snyder, Ancess, Einbender, Goodwin, and Macdonald, 1993, p.v.). Putting the learner at the 
center of school accountability means that schools are responsible for effectively engaging 
diverse learners, rather than being accountable for merely providing programs and delivering 
instruction regardless of the outcomes. 

Recognizing that learner outcomes will not improve unless they are directly addressed, learner- 
based accountability means that the entire culture of a school drives toward increasing 
student success. This view of accountability implies that as educators, we are responsible for 
demonstrating the impact of policies, programs, placements, and practices on learner 
outcomes— that we are accountable to the consumers of education, the children, as well as to the 
parents, community members, and other stakeholders that provide financial support for 
education. This notion of accountability is different from bureaucratic forms of accountability 
that focussed on compliance with procedures and directives. It "grows from a belief that school 
staff must look at and be guided by the results they produce in their students" (Yssledyke and 
Thurber, 1992). 

Some of the principles underlying learner-based accountability challenge traditional concepts of 
school organization. These principles emphasize designing curriculum based on what we want 
students to know and do, providing expanded opportunities for all students to learn and 
demonstrate what they are expected to know and do at a very high level, having high 
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expectations for all students, and shifting the reliance on norm-referenced standardized tests to 
the use of criterion-based performance assessments (Brandt, 1993). The curriculum is developed 
from the competencies students should demonstrate, rather than writing objectives that are 
derived from existing curricula. 

The premise behind organizing schools around desired standards of student performance is that 
there will be high expectations for all students, since instruction will ultimately focus on higher 
levels of learning and competencies for all students. This approach has significant implications 
for students with diverse needs, since it implies that "teachers will be much more focussed on the 
learning capabilities of their students and far less on covering a given amount of curriculum in a 
given time block." Focussing on outcomes creates an inevitable need for educators to 
accommodate the differences in learning rates in any group of students ". . .this paradigm 
challenges schools to establish delivery systems that can adjust to these time differences" (Spady, 
1992, 1993). The assumptions, principles, and practices that are the foundation for this new 
paradigm for schooling are dramatically different than those underlying the current paradigm, as 
shown in Figure 4. 

The principles driving the new paradigm hold great promise for ensuring positive educational 
outcomes for students with disabilities through inclusion. In its earliest form, inclusion was tied 
to the principle of "least restrictive environment" from P.L. 94-142. Early efforts were largely 
characterized by the physical movement of students out of self-contained classrooms and out-of- 
district placements into regular classrooms, where they might or might not receive support from 
a specialist in the classroom or resource room setting. While this approach may appear to 
achieve the goals of equal access and integration, effective inclusion requires that certain 
conditions be in place. These include a student-centered, success-oriented philosophy; 
instructional approaches that foster cooperative learning and ensure that a wide range of 



Page 28 

56 



Learner-Based Accountability 



FIGURE 4 



CURRENT PARADIGM FOR SCHOOLING 

• Schooling is organized around time: curriculum 
is "covered"; instruction is paced by the 
schedule, and assessment occurs at "unit" 
intervals. The "inputs" and process of education 
are emphasized over "product", or results. 
Schools accept the failure of a significant 
number of students. 

• Learning is organized around a standardized 
curriculum delivered in standardized time 
periods. Credentials are awarded based on 
"time-served", issued in "Carnegie Units." 

• The curriculum is derived from existing content, 
which is most often determined by textbooks. 
The curriculum is defined around a set of units, 
sequences, concepts, and facts. 

• Assessment is done at the end of instruction and 
is narrowly focussed on lower-level and 
fragmented (end-of-unit) skills that can be 
assessed through paper-pencil responses. 
Grades are based on a cumulative averaging of 
performance over a fixed period of time. Norm- 
referenced standardized test results are the basis 
of accountability, through which, by definition, 
half of all students in a norm group will perform 
below average. 

• School accountability is defined in terms of 
programs offered, attendance rates, and dropout 
rates; the number of students who are 
credentialed, and the results of standardized 
norm-referenced tests. There is minimal 
systematic monitoring of student progress on an 
ongoing basis. 

• School improvement focusses on improving the 
existing organization, e.g., by adding new 
programs, improving school climate, and 
increasing staff participation in decision making. 



NEW PARADIGM FOR SCHOOLING 

• The orientation to schooling is learning, 
achievement, and success. There is an emphasis 
on high levels of learning for all students. 
Diverse abilities, developmental levels, 
readiness, and learning styles are addressed so 
that all can succeed. The pace of instruction is 
based on learning, not how much content has to 
be "covered." 

• Learning is organized around what students 
should know and be able to do. Credentialing is 
based on student demonstration of proficiency 
in these knowledge and skill areas. 

• The curriculum is derived from standards that 
define what students should know and be able 
to do. Subject matter is "integrated" around 
"real-world" tasks that require reasoning, 
problem-solving, and communication. 

• Assessment is integrated with instruction and 
focusses on what students understand and can 
do. Assessment methods reveal students' actual 
competencies through demonstrations, 
portfolios of work, interviews, and other 
observational measures. Grades are based on 
culminating knowledge and competencies rather 
than an averaging of test scores. Criterion- 
referenced tests and performance-based 
assessments are used. 

• The school is accountable for ensuring and 
demonstrating that all students are developing 
proficiencies that represent high level standards 
for what students should know and be able to 
do. There is an emphasis on frequent 
monitoring of student progress. 

• School reform efforts are challenging and 
seeking to chartg'j the assumptions and practices 
that characterize how schools are currently 
organized. 

Center for Resource Management, 1994 
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resources, including computers are accessible to all students; a learner-driven curriculum that 
accommodates differences in student learning rates; appropriate in-class support; and 
opportunities for community-based experiences. Educational decision-makers involved in 
restructuring can also learn from methods emphasized in special education such as: 
individualized assessment and educational planning; modifying instruction and assessment to 
build on students' strengths; addressing post-school, "real world" requirements; planning for and 
supporting students through critical transition points; and involving parents in making decisions 
about their child's education. 

The growing emphasis on student learning and educational results that now drives improvement 
in all areas (including special education) represents an important opportunity to overcome a long 
history of fragmented programs and services. Recognition of the enormous diversity that our 
students represent — in abilities, learning styles, language ?jid culture, personal orientations, 
home situations, etc. — requires a level of individualization that traditional., "regular" education 
was not equipped to provide. By focussing on the levels of learning we want all our students to 
achieve, and by examining educational models in practice that demonstrate what is possible, we 
can move toward an empowering vision of education that achieves desired results. 

Learner-based accountability recognizes that what we really mean by success for all students is 
success for each student: the school is accountable for ensuring that each and every student is 
making reasonable progress toward acquiring the knowledge, skills, and orientations that 
represent the standards for what students should know and be able to do. Learner-based 
accountability also means that the school is responsible for tracking the extent to which students 
with particular characteristics or who are exposed to specific programs and practices are 
succeeding. The ability to systematically monitor the progress of individual and specific groups 
of students, then, becomes essential to ensuring success. The question, then, is no longer 
whether to establish accountability systems based on learner outcomes, but how . 
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A Learner-Based Accountability System 

The question of "how" to establish a learner-based accountability system is an important one 
because it raises issues about the purposes of accountability and the importance of ensuring that 
such systems are not narrowly focussed, but are responsive to the broader issues associated with 
ensuring excellence and equity for all students. Student outcomes have to be examined in the 
context of educational practice and the nature of opportunities that schools provide for students 
to learn — the question of wily outcomes appear as they do has to be addressed (Darling- 
Hammond, 1989, 1991, and 1992, Oakes,l989). An accountability system has to incorporate 
methods for accessing and using ongoing information about student performance and school 
effectiveness as well as processes that reinforce the use of effective practices. 

NCREST identified three building blocks of accountability as well as several key capacities that 
support the implementation of a comprehensive and effective accountability system. The core 
building blocks include: 1) a set of policies and procedures that encourage and support good 
teaching and valuable learning; 2) methods for regularly eliciting information that shows how the 
school is functioning for all students and that pinpoint what areas of the school context may be 
influencing the school's success or failure with individual students and various groups of 
students; and 3) mechanisms for rethinking and changing practices— in individual cases or in 
cases involving overall school functions— if students are not being well served. 

Underlying the NCREST accountability model are fundamental elements of school and 
classroom practices that have very positive implications for all students, including students with 
disabilities and those with a range of diverse needs. These practices include the following. 
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• A school organization that ensures attention to 
students' needs and problems and brings 
coherence to teaching and learning. 

° School policies and practices that "work both 
to provide an environment that is conducive to 
learner-centered practice and to identify and 
correct problems as they occur." 

° School operations that "heighten the 

probability that good practices will occur for 
students; reduce the likelihood that harmful 
practices will occur; and provide internal self 
correctives in the system to identify, diagnose, 
and change courses of action that are harmful 
or ineffective." 

• A set of processes for guiding practice and for 
using information to improve practice. 

° Vehicles for staff interaction, shared inquiry, 
and continued learning that strengthen practice 
and create opportunities for continual 
evaluation and improvement of teaching. 

• Forms of student assessment that reveal 
student strengths, talents abilities, and 
performance capacities. 



9 Feedback and assessment practices that 
prevent students from "falling through 
the cracks." 

• "Incentives to encourage all members of 
the school community to focus 
continually on the needs of students and 
the improvement of practice." 

• Systematic tracking of the progress of 
individual students or cohorts of students 
(rather than using aggregated averages of 
test score data) to determine the 
distribution of student achievement and 
school effectiveness. 

9 Evidence of the growth and progress of 
individual and groups of students over 
time — collecting and analyzing student 
performance data longitudinally to 
examine whether changes are the result 
of shifts in practice. 

• Mechanisms that help schools 
continually evaluate how well they are 
meeting students' needs. 



Creating Learner-Centered Accountability, NCREST, 1993 



Using Management Information System Technology to Support 
Learner-Based Accountability 

Putting learner outcomes at the center of school accountability is leading to more widespread 
recognition of the necessity for school-based capacity to systematically monitor student 
performance, and to evaluate the extent to which new approaches to curriculum, instruction, and 
assessment result in higher levels of student learning. However, while there is a growing body of 
educational research and practice literature on the many dimensions of school accountability at 
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both the policy and program levels, less attention has been given to how schools can develop the 
essential information system capacities that relate to learner-based accountability. 

For years, researchers and practitioners have emphasized the limitations of aggregated measures 
of student outcomes which do not support an understanding of whether specific groups of 
students are benefitting from their educational experiences. New visions of learner-based 
accountability require school-level capability to demonstrate the outcomes achieved by various 
groups of students by dis^ggiggMingjlata (sorting information) so that results can be correlated 
with pertinent student, program, and educational process variables; i.e., being able to obtain 
information about the performance of students with particular characteristics, the programs and 
practices to which they are exposed, and the outcomes they achieve. 

"The purpose for disaggregating student outcome data is to give the district and the individual 
schools a vehicle for evaluating their own effectiveness. The process seeks to identify the 
percentage of pupils in various subsets who achieve mastery of the essential learning at each 
grade level by program, course, school, etc. Through this analysis, a district and building can 
monitor whether students from all socioeconomic levels, different races, and both genders are 
mastering the essential student outcomes. Past experience verifies that such an analysis is one of 
the most critical steps in helping staff see the need for change. This analysis clearly shows 
whether the curriculum is being equitably learned by all students. " 

". . . Disaggregation is a practical, hands-on process that allows a school's faculty to answer two 
critical questions: Effective at what? Effective for whom? It is not a problem-solving process but 
a problem-finding process. " 

Lezotte and Jacoby, 1992 

The Center for Resource Management, Inc. (CRM) has conducted extensive research and 
development activities to determine the information system requirements of achieving school- 
level, learner-based accountability. This work involved hundreds of regular and special 
educators, parents, and community members, and drew from CRM's role in directing two major 
statewide school improvement initiatives in New Hampshire, from conducting school-level 
student outcome studies in more than 150 schools involved in school improvement and school 
restructuring projects in New Hampshire, South Dakota, and Colorado; and from evaluation 
studies that were funded under OSEP's State Agency/Federal Evaluation Studies (SAFES) 
Program A key issue that cut across all of these projects was how state-of-the-art management 

Gi 
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information system (MIS) technology can be used to build school capacity to systematically 
monitor student outcomes, demonstrate accountability, and implement continuous data-based 
program improvement. Areas that were examined include: 

1 . How schools currently access and use student performance data. 

2. The requirements of creating an integrated school-level database of student 
performance data and other pertinent information. 

3 . The types of information that regular and special educators believe should be 
included in an accountability system. 

4. The types of questions the system should address, and how data would be used by 
regular and special educators to improve student performance. 

5. How relational database technology can be used to enable schools to disaggregate a 
wide range of pertinent data, and to link student performance data with demographic 
and programmatic information. 

6. The training and technical assistance requirements of helping school staff develop 
skills that relate to effectively accessing and using information for monitoring, 
accountability, and ongoing improvement. 

The research process highlighted that focussing on student outcomes and developing essential 
information system capability represents a major culture change for most schools. Access to and 
use of performance data are extremely limited, and the way information is stored does not lend 
itself to easy access or analysis. Student performance data and other pertinent information is 
typically located in a variety of places, or is routinely destroyed at the end of a school year. 
Administrative software packages most commonly used by schools were not designed to 
function as accountability systems. They create schedules, generate report cards, produce school 
and grade-level attendance reports, and in some cases, grade distributions for specific courses. 
They were not designed to disaggregate performance data or to correlate performance data with 
demographic or educational process variables. Schools, therefore, can't link student performance 
data to specific programs, practices, and policies. This has made ongoing internal evaluation 
impossible, and has created a long-standing dependency on external evaluation; that is, without 
the assistance of external evaluators, schools have not been able to systematically assess the 
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effects of changes they have made in programs, policies, and practices on student performance 
over time. Thus, at a time when educators are expected to demonstrate stronger program 
accountability, most schools struggle to produce data to answer the most basic questions about 
the performance of specific groups of students. In short, demonstrating results and linking 
results to specific programs, classroom practices, and grouping polices requires integrated 
recordkeeping approaches and information technology that very few schools have. The current 
status of school-level capability to access and use information for program improvement and 
accountability is summarized in Figure 5. 

From discussions with more than 100 school teams that included both regular and special 
education administrators and teachers, parents, business and community leaders, and data 
processing personnel, several themes emerged that captured what practitioners and stakeholders 
want from a school-based information system. The system should: 
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Function to promote a school culture that values and uses information, in contrast to data 
systems that seem complicated or irrelevant to school staff. 

Function to focus school planning and improvement activities on ensuring success for all 
students. 

Provide teachers and administrators with the data they need to monitor student progress 
in formats that lend themselves to decision-making. 

Enable schools to communicate results to pertinent constituencies in formats that can 
easily be understood by parents, school board members, and community groups. 

Enable schools to produce data that address school accountability questions about the 
performance of specific students. 

Help schools conduct their own program evaluations and reduce dependence on outside 
evaluators. 

Be comprehensive — account for the wide range of student, outcome, and process data 
relevant to school-based monitoring and accountability. 

Provide methods and technology for creating an integrated school database, and for easily 
accessing, aggregating, and disaggregating data. 

Present data so that practitioners perceive issues systemically and not simplistically. 
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FIGURE 5 



Status of School Information and Recordkeeping Systems 

• School records aren'f integrated which makes the compilation of essential information 
burdensome. Student performance and other data are located in a variety of places. 
Administrative software packages create schedules, generate individual report cards, 
and provide limited school and grade aggregation. 

• Schools can't disaggregate data to monitor the performance of specific groups of 
students and the effects of school programs and practices on student performance over 
time. 

• Current systems don't have the capability to compile performance-based assessment 
data at multiple levels — classroom, grade, and school levels or for specific student 
groups. 

• Teachers and administrators don't have easy access to the data they need to 
systematically monitor student progress. 

• Schools can't produce data to answer accountability questions and to communicate the 
progress and success of specific groups of students to key stakeholders. 



Be sufficiently flexible to address individual school characteristics, priorities, and diverse 
information needs. 

Be interactive with the administrative software used by schools to avoid redundant data 
entry. 



These information system requirements represent powerful organizational capabilities for 
schools. They reflect key elements for bringing schools into the information age and for 
empowering school practitioners with information that directly relates to a mission of ensuring 
higher levels of learning for all students. To help schools acquire these capabilities, in 1990 
CRM began the development of a school-based management information system (MIS) that 
creates an integrated school-level database and provides almost unlimited capability to 
disaggregate student performance data. In developing the system, it was recognized that it would 
have multiple components, and its quality and utility would be tied to the extent to which certain 
practical realities were addressed. It would have to address school staff training needs, provide 
resource materials, prov'de forms and procedures for easily and efficiently accessing manual data 
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from school records, and utilize MIS software technology to provide adequate data analyses 
capability. It would have to take into account existing computerized databases, such as those 
used to generate report cards; the resource constraints and other realities faced by schools; and 
the types of report formats that facilitate interpretation and use by administrators, instructional 
staff, parents, school board members, and other groups. 

The challenge was to make the complex possible, and to produce an information system that 
could realistically be integrated into school operations to support ongoing data-based planning, 
monitoring, accountability, and program improvement. Schools do not need, nor will they 
accept, a system that is another "add-on" to an already burdened organization. Utilizing 
relational database software, a "user friendly" learner-based accountability system was developed 
that interrelates student, process, and outcome data elements, and provides school staff with the 
ability to not only address student outcomes, but also to address questions of why outcomes 
appear as they do. The system enables school administrators and instructional staff to use 
student outcome data in combination with other information to monitor student progress and 
improve school programs and instructional practices. Because of its emphasis on the use of 
information to improve student performance, the system is called the Student Profile System. 

The Student Profile System creates a central record of all pertinent information from schools' 
manual and computerized recordkeeping systems. Because it utilizes relational database 
technology, it does not displace schools' current computerized systems, but has the capability to 
import data already entered into these systems to eliminate duplicate data entry and to create an 
integrated database. The Student Profile System was designed to be as flexible as possible. 
Standard field names can be easily modified to allow for the use of user terminology which 
varies somewhat across schools for certain data elements. Based on input from hundreds of 
regular and special educators, customized output reports (which are called profiles) were 
designed to display data in formats that facilitate ease of use by school staff. While the Student 
Profile System includes a set of standard profiles, the profile formats are designed as "templates" 
that can be applied to various types of data and analyses. By having access to an integrated 
database, special education administrators, principals, and other school staff have extraordinary 
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information capability at their fingertips. The key features of the Student Profile System are 
summarized in Figure 6. 



FIGURE 6 



Key Features of the CRM Student Profile System 



THE STUDENT PROFILE SYSTEM: 

• Is a school-based information system that 
is customized to incorporate the 
characteristics, priorities, and diverse 
information needs of each school. 
Virtually any number of user-defined 
fields can be included to address the 
specific population, educational, and 
outcome variables the school wants to 
track. 

• Creates an integrated database of the 
information needed to monitor student 
performance. Data includes student 
demographics, school and classroom 
processes, performance assessments, 
absence, discipline, and dropout rates, 
grade distributions, and test scores. 

• Tracks the performance of special 
education students by school, grade level, 
disability, placement, specific special 
education services provided, number of 
service hours, and related services 
provided. 

• Allows easy access to student 
performance data for ongoing use by 
teachers and administrators. 

• Provides almost unlimited capability to 
disaggregate data so that schools can 
monitor the performance of specific 
student groups and the effects of school 
programs and classroom practices on 
student performance over time. 



Generates a wide array of reports (called 
profiles) on student performance in user- 
friendly formats for school staff and key 
stakeholders. 

Profiles performance-based assessment data 
at multiple levels — classroom, grade, and 
school levels — and for specific student 
groups. 

Is interactive with (can import data from) 
the administrative software used by 
schools. 

Allows schools to produce data that address 
school accountability questions about the 
performance of specific students. 

Helps schools develop site-based capacity 
to evaluate the effects of policies, 
programs, and practices on student 
performance over time. 

Focusses school improvement activities on 
increasing student achievement. School 
staff identify the factors that impact on 
performance, acquire performance data 
related to those factors, and develop the 
capacity to monitor progress and 
performance over time. 
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How Special Educators and Other School Staff Can Use a School-Based MIS to Support 
Student Learning, School Accountability, and School Improvement 

A school-based MIS designed to support learner-based accountability can be an important 
vehicle for helping special educators and other school staff develop shared understandings (a 
common language) about pertinent outcome indicators and achievement data. The installation 
and use of such a system should start with staff consensus about the outcome data available to 
them that provides information about student success. School staff quickly recognize that 
monitoring student progress means being able to document success and also being able to 
pinpoint where the trouble spots are — determining which students are not succeeding, and what 
factors seem to be associated with high or low achievement. It is important that school staff have 
opportunities to identify the factors they feel might impact on student performance. These 
factors become the data variables that are tracked through the MIS. They may relate to 
demographic characteristics of students, students' prior educational experience, and their current 
educational experiences, including curricular programs, specific courses, special programs, 
special education placement and services, instructional practices, assessment methods, and 
grading criteria. Monitoring student progress, therefore, means being able to relate outcome data 
to pertinent population and educational variables. Having this capacity is essential to linking 
outcome data to specific policies, programs, and practices. School teams see the value and 
power of this capability. Figure 7 depicts the range of information identified by school teams 
that have been incorporated into CRM's Student Profile System. 



O 
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The student variables represent groupings of students whose outcome data school teams want to 
monitor. For example, schools often want to examine the school performance of boys versus 
girls or of students from different ethnic groups. Some school teams have wanted to examine the 
performance of students from different economic levels or from different towns in a regional 
school setting. Because national studies are showing that students who read more on their own 
and spend more hours doing homework tend to achieve higher levels of proficiency than those 
who spend less time on these activities, some school teams have wanted to examine uV; 
relationship between student performance and these variables. 
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FIGURE 7 



The CRM Student Profile System 
Data Variables 



Student Variables 

• Gender 

• Ethnicity 

• Disability 

• Limited English Proficiency 

• Town of Residence 

• Economic Level 

• Family Structure 

• Social Service Involvement 

• Post School Aspirations 

• Cohorts by 

- Time in Work 

- Time on Homework 

- Time Spent Watching TV 

- Time Spent Reading 



School/Educational Process Variables 
Grade Level 
Sending School 
Early Education 
Prior Retention 
Chapter 1 Participation 
Special Education Placement, Hours of 

Service, Related Services, Classroom Support 
Other Program Participation 
Types cf Extracurricular Participation 
Course of Study 
Instructional Practices 
Class Size 
Grouping 



Outcome Variables 

Achievement Measures 

• Criterion-Referenced Tests 

• Performance Assessments 

• Norm-Referenced Tests 

• Other Tests 

Performance Indicators 

• Attendance Rates 

• Distributions of Grades or 
Other Performance Indicators 

• Discipline/Suspension Rates 

• Rates of Course Participation 

• Rates of Extracurricular Participation 

• Dropout Rates 



A relational MIS database also allocs educational/process variables to be linked to student 
performance. School teams can then^xamine-the outcomes of students who are in specific 
programs, placements, or courses of study. They can examine whether students who have been 
exposed to certain instructional practices or teaching styles are achieving higher levels of 
proficiency over time. The influence of class size or different types of grouping policies also can 
be systematically evaluated. An MIS is also an important tool for examining the effectiveness of 
inclusion for special education students. Special educators have wanted to monitor the 
performance of students in different types of placements and examine variations in performance 
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that may be linked to the amount of service hours provided. CRM is currently conducting a 
study under OSEP's SAFES Program that is examining the effects of various types of in- 
classroom support (such as special education staff co-teaching with classroom teachers, or the 
use of instructional aides) on the academic performance of special education students. The wide 
range of questions that can be answered by the data profiles generated through a school-based 
MIS is illustrated in Figure 8. 

A school team from a regional high school participating in the NH Special Education Program 
Improvement Partnership wanted to explore the relationship between student success and factors 
such as students' long-term and short-term goals, and hours spent in work or watching television. 
This school provides an example of how a performance-based approach to monitoring can 
directly involve students in examining the factors that may be influencing their success or failure, 



An Advisor Program was established to provide students with a relationship with an 
adult who could assist in developing long-terms goals and to set short-term grade 
average goals. Working with their advisor and parents, students selected long term 
goals ranging fioni 4-year college to the "world of work", and specified the grade 
average they would work to achieve. To ^ngage students in looking at factors that 
impact on success, the advisors administered a short survey that asked students how 
many hours per week they spend doing homework, watching television, and working. 
Students also were asked questions about their attitude toward school and self- 
confidence related to success in school. 

Because the Student Profile System was designed with student data as its base, long- and 
short-term goals and hours spent per week on various activities became p o pulation variables 
linked to eachstud ent. This allowed outcome data to be correlated and displayed for each of 
these variables, as well as for particular combinations (e.g., no homework or reading for 
pleasure), and the data profiles generated through the Student Profile'System illustrated the 
difference in performance for each of these variables. 

A high school team comprised of regular and special education staff and the advisors 
involved students in examining the data profiles. They held discussions to help students 
gain a better understanding of how certain factors within their control (e.g. homework, 
working, reading, and television time) may impact on performance. Students also 
examined outcomes achieved by students according to various short- and long-term 
goals. 
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FIGURE 8 



Types of Evaluation Questions Answered By 
A School-Based Student Profile System 



P opulation Trends 

1. What are the characteristics of the school 
population? Are there increases or decreases over 
time in the number of students with certain 
characteristics? 

2. What are the characteristics of students-with- 
disabilities and what trends are occurring over 
time for this population? 

Skill Development 

3. Does the performance of the school population 
on various assessments indicate that students 
have developed adequate competencies in the 
skill areas assessed? 

4. Are there particular skill areas where there is a 
high incidence of student success? Are there 
particular skill areas where many students are 
demonstrating below satisfactory performance? 

5. Are there notable differences in the perfor-mance 
of specific groups of students? 

6. Does the performance of students-with- 
disabilities on various assessments indicate that 
these students have developed adequate 
competencies in the skill areas assessed? 

7. To what extent are classroom practices such as 
cooperative learning, team teaching, and 
heterogeneous grouping having a positive impact 
on skill development? What are the 
characteristics of students who are succeeding 
with these practices? Not succeeding? 

Attendance. Discipline and Drop pin g Out 

8. Which groups of students appear to be at risk of 
school failure based on a combination of 
excessive absence and discipline problems? 

9. How many and what percent of the school 
population have been suspended or involved in 
disciplinary actions at least once? Three or more 
times? 

th 



10. Do the absence, discipline, or drop-out rates of 
any group of students indicate that there is 
problem that needs to be addressed for this 
group? 

1 1 . Are there trends in dropout data that indicate that 
particular groups of student^ appear to be at risk 
of not achieving future employment? 

Course Participation 

12. What is the participation rate of various groups 
of students in specific courses in the subject areas 
of Language Arts, Mathematics, Science, Social 
Studies, and Vocational Education? 

13. What is the participation rate of students with 
disabilities in lower level courses? In advanced 
courses? 

Grade Performance 

14. How is the student population performing in the 
major subject/skill areas of Language Arts, 
Mathematics, Science, and Social Studies at all 
grade levels? What is the distribution of grades 
in these major subject areas? 

15. Do grading patterns suggest any inconsistencies 
in criteria across subject areas? Classrooms? 
Within or across departments? At key transition 
points? 

16. Does the grade performance of any group of 
students indicate they are at risk of cumulative 
academic failure? 

17. How are students with disabilities in specific 
placements or programs progressing? Are 
students who are in integrated settings achieving 
satisfactory grades? 

Participation in Extra-Curricular Activities 

18. What is the participation rate of students in 
various school activities? 

19. Are there notable differences in the participation 
rate of specific populations? 
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How a School-Based MIS Supports School-Level Implementation 
of the NCEO Framework 



As discussed in a previous section, the National Center on Educational Outcomes (NCEO) 
developed a conceptual model of outcome domains, related outcomes, and possible indicators for 
each outcome. Outcome indicators are the actual data that schools can use to demonstrate the 
extent to which various outcomes have been achieved. For example, for the domain of Academic 
and Functional Literacy, the model proposes that students should demonstrate competence in the 
following five areas: communication; problem-solving strategies and critical thinking skills; 
math, reading, and writing skills; other academic and nonacademic skills; and using technology. 
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Schools seeking to monitor and demonstrate student progress against the above indicators must 
deal with measurement issues as well as accountability issues. The measurement issue is tied to 
the assessment methods used by schools through which students actually demonstrate 
competence. As discussed previously, there is widespread recognition that the use of norm- 
referenced tests and the narrow types of textbook-driven "end-of-unit" or "end-of-semester" 
assessment methods used by schools do not adequately assess student competencies, nor do they 
measure higher order communication, reasoning, and problem-solving skills. Thus, while 
standardized test data and grade performance data provide some evidence of student 
competencies, the evidence is insufficient. Monitoring and communicating to parents and other 
stakeholders the percent of students who actually demonstrate academic and functional 
competencies in the five areas identified in the NCEO model will require that statewide testing 
programs and schools become more proficient in the use of alternative forms of assessment that 
reveal students' real performance capacities. Current reform efforts are already showing this 
shift. Statewide assessment programs are rapidly shifting to the use of criterion-referenced 
measures. Schools are shifting toward teaching and learning practices that integrate assessment 
with instruction, focus on what students understand and can do, and draw upon the use of 
portfolios, performance demonstrations, and other observational measures to assess student 
proficiency. 
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The accountability issue is tied to the extent to which school information systems incorporate a 
sufficient array of student performance data that includes criterion-referenced, performance- 
based assessment data, and are capable of compiling and disaggregating data so that schools can 
monitor and demonstrate the progress of specific groups of students. Current information 
system technology can be designed to provide schools with these essential capabilities. 



Figure 9 on the next page illustrates how a system such as the Student Profile System acts as a 
vehicle for helping schools implement conceptual models such as the one developed by NCEO 
by allowing schools to develop a longitudinal database to systematically track the percentage of 
students (for the total school population and designated sub-populations) who develop desired 
competencies over time. It enables regular and special educators to determine: the extent to 
which students in inclusive classroom settings are developing competencies that represent 
emerging visions of what students should be able to know and do; whether the special services 
provided to students enhance the development of desired competencies; and whether particular 
classroom practices have a positive impact on student success. Information system technology 
can effectively support an educational approach that emphasizes progress over process, and 
equity of opportunity and results. As such, it can support reform initiatives aimed at ensuring the 
success of each student. 
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FOREWORD 



This report is the result of a study done under Project FORUM, a contract funded by the 
Office of Special Education Programs (OSEP) of the U. S. Department of Education and located 
at the National Association of State Directors of Special Education (NASDSE). Project FORUM 
carries out a variety of activities that provide information needed for program improvement, and 
promote the utilization of research data and other information for improving outcomes for 
students with disabilities. The project also provides technical assistance and information on 
emerging issues, and convenes small work groups to gather expert input, obtain feedback, and 
develop conceptual frameworks related to critical topics in special education. 

The purpose of this analysis is to update and add to the information contained in a 1992 
Project FORUM analysis of the results of a 1992 survey on state practices in compliance 
monitoring that also contained input from the Second National Conference on Monitoring held 
that year. A Third National Conference on Monitoring was held in November, 1994, and it 
involved the distribution of the results of another survey on state monitoring policies and 
practices that had been conducted by the Regional Resource Centers. This activity was included 
in Project FORUM's tasks for the 1994-95 year to provide State Directors of Special Education 
and others with important and timely information on this critical issue and especially to inform 
the debate that will be occasioned by the pending reauthorization of the Individuals With 
Disabilities Education Act. 
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STATE MONITORING PRACTICES: AN UPDATE 



INTRODUCTION 

The pending reauthorization of the discretionary programs of the Individuals With 
Disabilities Education Act (IDEA) has focused interest on the implementation of the act over the 
past twenty years. This analysis is intended to provide an overview of current policies and 
procedures that states use to oversee that implementation in their local school districts. 

In December, 1992, Project FORUM issued a report entitled Analysis of State Compliance 
Monitoring Practices in which the information gathered at the 1992 National Conference on 
Monitoring and the results of a survey of state monitoring practices completed in that year by the 
Regional Resource Centers (RRCs) were analyzed and discussed. In November, 1994, the RRCs 
convened the Third National Conference on Monitoring and once again surveyed state monitoring 
practices. This documents is an update of the earlier report on state monitoring practices and is 
based on data from the more recent conference and survey. (Appendix A contains a copy of the 
survey form; a chart of survey results on selected items is included as Appendix B.) 

This report consists of a brief background on the law and its requirements for monitoring, 
some child count statistics, and a description of specific monitoring practices in the states 
including the following: use of the self-study as part of the process, the composition of onsite 
visiting teams, cycles of review, the incorporation of measures of effectiveness and outcomes into 
the monitoring process, and other changes adopted recently by states. Also included is a brief 
profile of the unique aspects of five state monitoring systems. The report concludes with a 
discussion of current trends and proposals to improve monitoring practices at both state and 
federal levels. 

BACKGROUND 

Federal and State Monitoring Systems 

The passage of P.L. 94-142 (Education of the Handicapped Act, since renamed the 
Individuals with Disabilities Education Act or IDEA) in November, 1975 marked the beginning 
of a far-reaching change in the interaction between State Departments of Education and the Local 
Education Agencies (LEA) or Intermediate Education Units (IEU) that deliver direct services to 
students with disabilities within the state. The law granted federal funds for providing special 
education services to students with disabilities and mandated new responsibilities for States to 
monitor how localities provide those services. Specific sections of IDEA and the regulations 
implementing it, as well as requirements in the Education Department General Administrative 
Regulations (EDGAR), contain responsibilities for Federal oversight, but §1412 of the Act 
contains the general provisions that authorize State monitoring of the provision of education to 
students with disabilities: 
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The State educational agency shall be responsible for assuring that the 
requirements of this subchapter are carried out and that all educational programs 
for children with disabilities within the State, including all such programs 
administered by any other State or local agency, will be under the general 
supervision of the persons responsible for educational programs for children with 
disabilities in the State educational agency and shall meet education standards of 
the State educational agency. [20 U.S.C. 1412(6)] 

Since 1975, both the U.S. Department of Education and State Departments of Education 
have developed monitoring systems to assess compliance with applicable statutes and regulations 
pertaining to programs and services for students with disabilities. Originally termed the Program 
Administrative Review and now referred to as a multi-faceted program review process, the series 
of activities used by the Office of Special Education Programs, Division of Assistance to States 
(OSEP/DAS) to determine the extent to which a State is in compliance with IDEA and related 
requirements include.; an examination of documents (State Plan, annual performance reports, 
policies, etc.) and an onsite verification of the implementation of federal and state laws and 
regulations. 

As described in the Sixteenth Annual Report to Congress (U.S. Department of Education, 
1994, p. 173), monitoring procedures used by OSEP to evaluate each state's implementation of 
IDEA have evolved as a result of changes in the law and modifications in the strategies states use 
to meet their responsibilities under the law. Since their inception, however, monitoring 
procedures have involved both a documentation review and a visit to the state by a team of OSEP 
staff. Currently, OSEP staff carry out a pre-visit set of activities that include one or more public 
meetings held in the State and preparation for the onsite visit. During their week-long presence 
in the State, OSEP staff review state documents and interview state officials, visit selected 
schools and other public agencies, review IEPs and interview personnel responsible for programs 
for students with disabilities. As a part of each compliance review, OSEP assesses the state's 
procedures for monitoring local education agencies and intermediate education units. The entire 
process results in a written report that cites non-compliance and a corrective action plan to bring 
state policies and procedures back into conformity with federal requirements. 

In the past, a draft of the monitoring report compiled by OSEP was submitted to the SEA 
for review with an opportunity for discussion of its contents before it was made public. 
However, in a memorandum (OSEP #94-19) issued on April 28, 1994, OSEP advised the states 
that, in accordance with a Presidential directive to ensure compliance with the Freedom of 
Information Act (FOIA), OSEP would henceforth make the Draft Monitoring Report available 
to the public in response to a FOIA request at the same time that it is released to the state. 

OSEP requires that states include in their monitoring system the procedures necessary to 
determine their LEA's compliance with every state and federal requirement. Although there are 
variations among state monitoring systems, they have all developed a structure simi'ar to the 
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federal approach. Some states add special components or vary the procedures as will be 
discussed below. However, in general, uie monitoring process in states includes a review of 
records such as the LEA and/or IEU funding applications, complaint investigations, self studies, 
child counts and other data, some type of onsite verification of the implementation of policies and 
procedures, and the prescription of a correction process for any identified deficiencies. 

Factors Affecting State Monitoring Systems 

The design of monitoring procedures in a state is influenced by a number of factors 
including the size and geography of the state, the administrative structure, the number of school 
districts, the number of other entities that must be monitored, and the number of students with 
disabilities who receive special education. The RRC survey that will be discussed in detail 
throughout this report included items that captured some of these factors, and a summary of those 
data is included here to provide a basic context for the description of state monitoring systems. 

Child Count: The total number of children found eligible and served by each state under 
its special education programs and services is collected in each state as of December 1 st of each 
year. 1 The distribution of federal funds under IDEA is based on that child count. This number 
has increased every year since the implementation of IDEA. It is subject to audit and is often 
revised. However, in order to provide a general frame of reference for the size of state programs 
and the support provided by the federal government, the child count used for official notifications 
of funding issued by OSEP for the 12/1/91 and the 12/1/93 were reviewed and compared. The 
12/1/93 total for each state is included in the survey data chart in Appendix B. 

The IDEA child count increased by 362,205 between 12/1/91 and 12/1/93. The totals 
were 4,682,604 in 1991, and 5,044,809 in 1993 representing a 7.74 percent increase over that two 
year interval. Only three states registered a decrease during that period — Vermont (9.2%), 
Pennsylvania (1.8%), and West Virginia (0.4%), while three others— Alabama, Kentucky and 
North Dakota— had increases of less than one percent. The highest increase occurred in Nevada 
(23.4%), and the next highest set of increases ranging from 13 to 14 percent were in Georgia, 
Arizona, Alaska, Louisiana, New Mexico and New York. 

Entities Monitored: Because the IDEA makes the state education agency (SEA) 
responsible for delivering a free appropriate education to students with disabilities, the SEA must 
have a plan to monitor the compliance of all providers with federal and state law and regulations, 
including school districts and others. The number of entities monitored refers to LEAs (local 
education agencies or school districts), IEUs (intermediate education units), other state entities 
(such as institutions maintained by other state agencies), and private programs or service 
providers. The child count and entities in each state are indicated in Table 1 . 



1 Each state also counted and received separate funds for children served under the "State Operated Programs" section 
of Chapter 1 of the Elementary and Secondary Education Act until 1994 when the funds available under that law were 
transferred into IDEA and those children were then included in the IDEA count. 
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This listing of child count and entities to be monitored communicates some indication 
of the size of a state's monitoring job, but there are differences among states that must be 
noted in connection with these demographic data. In some states such as Illinois and 
Michigan, the monitoring of LEAs is organized through intermediate units and visits are 
planned on the basis of the cluster of districts that are part of each unit. The monitoring of 
private schools also differs: either they are monitored separately, or they are included as a 
part of the LEA or IEU in which they are located, or they are under the control and oversight 
of other state agencies. 

CHARACTERISTICS OF STATE MONITORING SYSTEMS 

A cop of the survey instrument, Profile of State Monitoring Systems, used by the 
Regional Resource Centers to gather information for the 1994 National Monitoring 
Conference, is contained in Appendix A. It included the same items as the 1992 survey with 
some added level of detail such as the breaking out of the total entities to be monitored into 
categories, and the addition of five new items: two about Part H, two about monitoring 
beyond requirements to include program effectiveness and student outcomes, and one on 
interagency agreements. 

Responses were received from 49 states and nine of the 10 federal jurisdictions that 
are subject to the requirements of IDEA. The Wisconsin SEA could not respond to the 
survey because their monitoring system was undergoing a complete revision and decisions had 
not yet been finalized on many of the areas covered by the items. 

It was not possible to analyze data reported for the three items that pertained to SEA 
staffing. It was learned through phone contacts with SEA personnel that SEA structure and 
staff responsibilities posed problems for those who responded, thus yielding what appeared 
to be discrepancies in the reported data. A variety of factors were found to have influenced 
responses to the staffing items. In many states, staff have assignments that cover more than 
one program area, so it is difficult to isolate the exact amount of time that is spent on 
monitoring. This differentiation is made even more difficult by the variations in what is 
defined by each state as part of the monitoring process. In some states, those who handle 
complaints are considered part of the monitoring staff, while in others this is a separate 
function. In addition, many SEAs see an overlap between monitoring and technical assistance 
for their LEAs and feel that there should not be a hard line drawn between these two 
functions. Given this lack of consistency in definition, no attempt was made to draw any 
conclusions from the staffing data. 

Although there is much general similarity between state and federal monitoring 
systems, there are differences in the specific components that each state chooses to include 
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in its system. The variations that were discussed in the first report in 1992 included the 
composition of onsite monitoring teams, the use of focused/targeted monitoring, monitoring 
beyond federal requirements, and some other specific aspects of the monitoring process in 
states. The next section of this report covers issues related to two characteristics of 
monitoring systems that were not discussed in detail in the previous report— monitoring cycles 
and LEA self-monitoring— followed by data related to the revised items on monitoring 
beyond federal requirements. A brief update on some of the other characteristics of state 
monitoring systems discussed in the previous report is also included. 



Monitoring Cycles 

States usually divide the entities to be monitored into groups determined by the 
monitoring cycle which can be defined as the number of years between onsite visits that the 
state has adopted for its monitoring program. The average length of state cycles is 4.6 years 
and they range from a low of two years in Alabama, to a high of eight years in Ohio. 

There are almost as many patterns for monitoring as there are states. Some states have 
designed a schedule of different activities related to monitoring that take place in each year 
of the cycle. The Colorado system is an example of this annually structured approach: 

Year 1 : LEA child count audit through a record review to check compliance with 

eligibility data and federal count reports; 
Year 2: LEA submits a comprehensive plan by the LEA, and SEA reviews data 

collated on the LEA concerning complaints, hearings, and other 

matters; 

Year 3: Comprehensive onsite visit to check compliance, determine local needs 
and provide technical assistance regarding program quality and 
effectiveness; 

Year 4: Follow-up technical assistance by SEA for areas identified as corrective 
actions and compliance concerns during the onsite visit; and, 

Year 5: Targeted onsite visitation to assure that the LEA has completed all 
corrective actions. 

Some states have a variety of the multi-year schedule or another time design based on 
the various components of the monitoring system, while others concentrate most of their 
monitoring activity relative to each LEA into the year in which the district will be visited. 
However, even in states with the single year focus, follow-up to a corrective action plan and 
verification of the corrections usually extend beyond the year of the onsite visit. Many states 
describe monitoring as an ongoing, continuous activity. 
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A total of fourteen states reported changes in their monitoring cycles since the 1992 
report. As Table 2 indicates, 12 states increased the number of years, while only two made 
their cycles shorter. 

Table 2 

Changes in Number of Years in 
State Monitoring Cycle 
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Shaded rows = decreased cycles 

LEA Self-Monitoring 

According to survey responses, 25 out of the 49 responding states include self- 
Rionitoring by LEAs as pari; of their process. For some, it is a voluntary component of the 
process, but for others — or in certain circumstances — it is mandatory. Some states include 
a type of self-review within the LEA application for federal funding or in a mandated 
program plan that each LEA must submit to the state. However, states that include a self- 
study in the monitoring process consider it a separate element and provide specific forms or 
directions for the LEA to use in preparing its content. A completed self-review is always an 
integral part of the state's monitoring of the district. 
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The California Coordinated Compliance Review that monitors all specially funded 
educational programs including, in addition to special education, Chapter I, vocational 
education, adult education gender equity programs and other school-based coordinated 
programs, is an example of a voluntary self-review as a monitoring component. According 
to state training materials, the purposes of the self-review are to have the LEA take 
responsibility for reviewing specially funded programs for compliance, take corrective action 
prior to state validation review, and identify areas in which assistance is needed from the 
Department (California Department of Education, 1994). The benefits mentioned as derived 
from the self-review since its initiation in the mid-80s include: 

o The LEA gains an opportunity to correct identified non-compliance problems prior 
to the state validation review; 

o The state validation review process is shorter when the local entity completes a self- 
review; 

o LEA staff and community gain a better understanding of state and federal 
requirements; and, 

• LEA staff and community feel better prepared and less anxious about the state 
validation review. 

Self-evaluation is a required part of the Florida monitoring system: the LEA must 
undertake a review of critical components of their special education program. Work papers 
are provided to the LEA, which must "self-evaluate" at least one student record for each 
program area. The names of the students whose records are to be self-monitored are selected 
by the state, and the LEA must provide a summary of their findings from each file. The 
LEA's "Report of Self-Evaluation" is included as a part of the state's monitoring report. 

A full year of self-study is part of the Kentucky monitoring system. The state 
provides training to encourage districts to spend an entire year evaluating their programs and 
completing the self-study that must be submitted by May 1 st . The site visit takes place in the 
following school year, and the same instrument is used by the state to verify LEA 
compliance. 



Monitoring Beyond Federal Requirements: Program Effectiveness and Student Outcomes 
as Part of the Monitoring Process 

The 1992 RRC survey item that asked states if their monitoring went "beyond legal 
obligations" was expanded in the 1994 survey to two items that asked if the state went beyond 
legal obligations in the areas of program quality/effectiveness, and student outcomes/results. 
While 19 of the 49 responding states indicated that they included program quality or 
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effectiveness, only five said that they incorporate consideration of student outcomes/results 
in their monitoring process. 

Some of the 19 states that responded positively to the item concerning program 
effectiveness as a part of their monitoring system provided brief details in the narrative 
component of the RRC Survey Report. For example: 

° Idaho describes the major activity in year three of the monitoring process as an 

exemplary programs/effectiveness review that involves an onsite visit by personnel © 
from other LEAs to verify the district's application for recognition. 
° Nevada also includes a program effectiveness review as a required part of its 
monitoring system, suggesting that districts use a stakeholder model. Although 
Nevada districts can choose to use a model that differs from the one recommended 
by the state, they must submit a report of their program effectiveness activities to CD 
the SEA every three years. 
° Louisiana also includes a quality indicators component as a part of the self-study 

process, but completion of this document is voluntary for LEAs. 
° Michigan has piloted a set of quality indicators based on monitoring standards at 

the request of one of its intermediate school- districts. The process is being refined © 
and will be incorporated as an optional component of the Michigan system. 
• Alabama noted that, when SEA staff identify a promising practice, the LEA is 
requested to prepare a summary that can be shared with other districts — a 
component of the review system known as "PSST or Practices Supporting 
Successful Teaching." ® 

The five states that responded affirmatively to the item concerning student outcomes 
were California, Idaho, Maine, Texas and Vermont. Details about these items for California 
and Texas are contained in the next section that profiles distinctive monitoring systems in a 
few selected states. The other states are all in the process of trial implementation of various © 
strategies to add this aspect to the monitoring process. For example, in four supervisory 
unions monitored in the 1993-94 school year, Vermont piloted a student outcomes component, 
and this segment will be added to three additional monitoring visits in the 1994-95 cycle. 

Maine responded affirmatively to this item even though they do not gather specific © 
data on outcomes as a part of monitoring. The SEA staff feel that the major purpose of 
monitoring, especially the examination of IEPs, addresses outcomes issues. Although it is not 
officially a part of their process, Maine's monitoring staff advised that they do discuss issues 
relative to the completion of goals, movement to less restrictive placements, and other types 
of outcomes information on monitoring visits to districts. 
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Composition of Monitoring Teams 

Although States are not compelled by Federal statute or regulations to make formal 
compliance visits to LEAs or ESUs, the actual implementation of policies must be verified. 
As mentioned above, states have generally adopted a monitoring system similar to the federal 
process and that includes an onsite visit to verify the implementation of federal and state 
policies. Every state uses at least one representative of the SEA on each team, but the 
teams in 16 states are composed exclusively of SEA personnel. Aside from SEA staff, states 
use staff from other school districts such as special education administrators or teachers (LEA 
peers), parents from other LEAs, and/or others such as university faculty or contracted 
individuals. Table 3 displays the responses to the RRC survey items concerning types of 
individuals included on state teams. States that include all possible categories on their teams 
are shaded in the table. 



Table 3 

Composition of State Monitoring Teams 



STATE 


SEA Staff 


LEA Peers 


Parents 


Others 


AL 


V 








AK 


V 






V 


AR 


V 


V 






AZ 


V 


V 






CA 


V 


V 






CO 




III v iil 


Bill IIS 




CT 


V 








DE 


V 






V 


FL 


V 








GA 


V 








HI 


V 


V 






IA 


V 








ID 


■ V 


■ '• V 


V 


V 


1L 


'•• . v '■- " 




... . ' - 


• . v 


IN 


V 


V 




V 


KS 


V 


V 




V 


KY 


V 


V 




V 


LA 


V 


V 




V 


MD 


V 
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STATE 


SF.A Staff LEA Peers 


Parents 


Others 


MA 


V 








ME 


V 


V 




V 


MI 


V 


v 1 


V 


MN 


V 


V 




V 


MS 


V 








MO 


V 


V 






MT 


V 


V 




V 


NC 


V 


V 






ND 


V 






V 


" m 


V 






V 


NM 


V 








NV 


V 


V 




V 


NH 


V 


V .. 




V 


NJ 


V 








NY 


V 








OH 


V 








OK 


V 








OR 


V 


V 




V 


PA 


V 


V 




V 


RI 


\ v 




V 


■ V ■ 


SC 


V 








SD 


V 








TN 


V 








TX 


V 


V 






UT 


V 






V 


VT 


V 


V 




V 


VA 


V 


V 




V 


WA 


V 








WV 


V 








WY 


V 






v 
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The survey results show that some change has occurred in these data between 1992 
and 1994. Three states— Maryland, Massachusetts and Wyoming— discontinued using LEA 
peers, while Kentucky added them to their teams; six states that had included parents no 
longer do so— Indiana, Louisiana, Massachusetts, Michigan, Minnesota and Montana— while 
Illinois added parents to their teams; seven states that had not previously used others on their 
teams— Alaska, Kentucky, Maine, Nevada, North Dakota, Virginia, and Wyoming— began 
including them, while six states—California, Maryland, Massachusetts, Missouri, Oklahoma, 
and South Dakota — no longer do so. 

Usually, when an SEA includes LEA peers on a monitoring team, the appointment is 
for one visit or, as in Minnesota, the LEA peer is entered on a list of available personnel to 
be called on periodically for a monitoring assignment. Maine uses a different approach. 
The state "buys the contract" of three LEA administrators for a year and, in conjunction with 
one SEA representative, they comprise the monitoring team. The LEA administrators remain 
on their local payroll, so there is no interruption in their employment benefits. The SEA sees 
this strategy as very effective with the only drawback being the loss of the contracted people 
after they have acquired expertise in carrying out the monitoring process. 

The methods used by states to train non-SEA team members vary. For example, 
Indiana offers a structured two-day training course four to six times a year and maintains a 
list of individuals who are trained and available for assignment to teams. This formal 
approach was tried and rejected by Colorado in favor of a more individualized approach in 
which the SEA program person who chairs the team is responsible for preparing the LEA 
personnel and parents who comprise the membership of each team. With assistance from the 
Great Lakes Regional Resource Center and input from other states, the Minnesota SEA 
designed a system that reflects their commitment to the use of a broad-based cadre of peer 
monitors. Individuals must apply to this program, and selection criteria are based on the 
SEA's goal of having a diverse pool of peer monitors who represent all types of 
constituencies. The program, first implement in the fall of 1989, includes an intense training 
component for peer team members. 

Non-SEA team members almost always have a specific role in the work of the team 
even if they are not involved in every aspect of the team prt ;ss. The SEA representative 
is usually the team chairperson and is responsible for final compliance decisions and 
preparation of the report. LEA peers and other education professionals are usually assigned 
tasks that involve interviewing district personnel and garnering information from other sources 
about the district's programs and services. The role of parents on a seam is usually focused 
on the assessment of parent-related issues in the district being monitored. 

States that use non-SEA members on teams expressed strong support for this approach 
during the 1992 data collection. State personnel used terms such as 'field colleague* and 
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'significant others' to refer to these added individuals which communicated the importance 
attached to their participation. States that provided input on this issue cited many advantages 
and almost no disadvantages when asked for their perception of this practice. State 
monitoring staff said the use of individuals other than employees of the SEA on onsite 
monitoring teams helps to: 

focus the process on how a district serves children rather than just whether they meet 
the letter of the law; 

de-emphasize the negative and adversarial connotations that the monitoring process 
usually carries; 

supplement a small SEA monitoring staff allowing for more comprehensive and 

accurate evaluation of compliance in districts; and, 
provide positive public relations for the SEA with school personnel, parents and other 

community constituencies. 

State monitoring staff singled out the use of LEA peers on monitoring teams as 
especially beneficial. They said that LEA peers: 

• help to forge a partnership, changing the relationship between the SEA and the LEA 
or IEU; 

make the monitoring process more tolerable to local units because the peers are closer 
to current field experience and frequently have more credibility with school staff 
than SEA personnel; 

provide technical assistance to their colleagues even if only informally and foster the 
development of professional networking for program improvement; and, 

obtain a better understanding of compliance and learn ways to improve their own 
district's program. 

Most SEA respondents felt that there were no disadvantages to the use of non-SEA 
members on teams. A few cited some problems that could occur, but in all cases they were 
not seen as significant. The difficulties (usually expressed as only theoretically possible and 
not actually encountered) mentioned by State monitoring personnel were: 

some non-SEA individuals do not understand compliance as a legal concept and this 

can interfere with their conclusions about LEA practices; 
the presence of non-SEA members on the team can lead to a 'we-they' division if a 

team member appears to 'side with' the LEA or IEU being monitored; 
sometimes non-SEA members are 'too tough' or 'too lenient' when it comes to 

making compliance judgments; 
some people have strong biases or personal issues that interfere with their ability to 

be objective about the LEA/IEU; and, 
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managing a team composed of different types of members is not as easy as dealing 
with SEA peers who are familiar working partners. 



Sanctions for Non-Compliance 

The 1992 report noted that there was very little differerce among states in the 
application of sanctions for LEA non-compliance. In the 1994 survey, only 17 of the 49 
states replied affirmatively to the item, "We use sanctions other than fiscal sanctions." This 
number is seven fewer than the 1992 survey. Actually, there is little evidence that states use 
formal sanctions of any kind Or compliance problems. States acknowledge their power to 
use fiscal sanctions, but this consequence is rarely invoked. Rather, SEAs negotiate the 
correction of problems, applying pressure uj-ough other means such as close follow up, 
additional onsite visits, and the targeting of technical assistance. Most states said that, 
although the threat of a loss of funds is an ever-present motivation for districts to comply, 
they do not consider the actual withholding of funds to be an appropriate sanction except in 
cases of continued refusal to implement corrective actions. However, many states do apply 
fiscal sanctions for any errors found in an LEA child count in which ineligible students had 
been counted for funding purposes. 

One exception to this trend was found in North Carolina where the monitoring process 
was changed in the last year to combine the headcount audit and the program compliance 
visit. Monitors now have the authority to require payback of state and federal dollars from 
LEAs when a compliance violation is found, and the penalties are applied automatically. 



PROFILES OF SELECTED STATE MONITORING SYSTEMS 

Arizona 

Arizona has used a variety of strategies in its monitoring system. About 20 years ago. 
the SEA used teams of LEA peers in its monitoring process and this was popular with school 
personnel. LEA team members found that serving on teams helped them assess their own 
compliance status and provided them with new progiam ideas. However, the SEA felt that 
this approach did not provide enough compliance data even though it yielded lots ot 
information on program improvement. Teams were then limited to SEA staff specifically 
hired for monitoring, but that approach was also found to be unsatisfactory because of rapid 
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turnover of staff. Self-monitoring and "quality reviews" were used for a time, but this also 
was found to lack adequate validation of compliance. 2 

Five years ago, Arizona implemented a new monitoring system, the Collaborative 
Program Review (CPR), that is designed to be a part of overall education reform and school 
improvement. The rationale for the new system as stated by the SEA (Arizona Department 
of Education, 1993) is the need to balance the current procedural focus of special education 
accountability with an increasing emphasis on outcomes for students with disabilities. In 
addition, the SEA felt that the monitoring process excluded school district staff from 
participation and did not recognize the changing roles and responsibilities for both state and 
local educators and administrators. 

Under the new system, a district may opt to participate in the CPR as an alternative 
to the standard monitoring review that is still available. Once a district is chosen, they are 
awarded a grant of $2,000 to make staff available to be a part of the newly-structured 
monitoring team that includes representatives from all general and special education sections 
of the LEA. This working group is trained by the SEA to plan and carry out the actual 
monitoring. They must develop a plan that includes the required components of the standard 
monitoring review and at least one element not related to compliance. SEA staff assist in the 
onsite visit, but the LEA team carries out all the monitoring activities including the writing 
of the report. Then, a program improvement plan must be developed in addition to a plan 
for correcting any non-compliance. 

SEA staff agree that the CPR is not easier or cheaper to do than the traditional process. 
However, the involvement of general and special educators at the local level has had 
important benefits for me districts. The flexibility of the process makes it more responsive 
to individual district differences, and local special education administrators report more 
cooperation with general educators. In addition, there is increased recognition of the 
importance of compliance as a result of the new procedures. 



California 

In California, special education is but one component in a combined compliance 
procedure called the Coordinated Compliance Review (CCR) that addresses all specially 
funded programs including migrant education, adult education and other categorical programs. 



^orth Carolina personnel also described their experience with "quality reviews" as problematic. Districts that had 
agreed to undergo such voluntary reviews were exempted from a compliance review. Among the reasons this practice 
was elirr ir-ated by North Carolina was the opinion of many local administrators that they needed the forceful consequences 
of compliance monitoring to use as leverage to obtain or protect the resources tney needed for services for students with 
disabilities. 
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The visiting team has only one member representing special education and that place may be 
filled by either a SEA specialist or a professional educator who has been trained in the 
compliance process. A single report is issued, of which special education is only one part. 
The CCR occurs in two phases: the LEA self-review (described above on page 9) and the 
state validation review. 

Two variations of the process are available for school districts that meet certain 
criteria. A screening review that involves a one or two-day visit by SEA staff is used to see 
if the district's self-review is accurate enough to be accepted as the official compliance 
review. To qualify, a district must have an enrollment of less than 20,000, have had a 
previous noncompliance rate of less than 8.62 percent, and have addressed past 
noncompliance findings. Document review is the other variation available to very small 
districts. A meeting is held with the district at a regional office to verify the self-review 
through written records. In both cases, a full validation review would follow if the 
abbreviated procedures suggest any significant deficiency or non-compliance. 

LEA training and extensive materials are provided by the California SEA (California 
Department of Education, 1994) for the CCR. Two types of instruments have been devised: 
one to cover items that apply to more than one program, and a second that covers items 
specifically related to only one program. Compliance items are grouped under key strategies 
for each program goal, and guidelines are given on how to test for that particular compliance 
item and what specific materials or activities to review. 

The CCR self-review includes a program item about student outcomes that, while not 
a compliance issue under special education, is nonetheless a required element. The particular 
goal is stated in the Training Manual as follows: "Multifunded students receive a coherent and 
coordinated program which enables them to learn the district's core curriculum." A suggested 
form is provided for the LEA to submit a summary of specific data as evidence of student 
learning. A positive evaluation is concluded when the district shows that students with 
disabilities are meeting the district's grade-level expectations in the core curriculum or that 
they are making significant academic gains that will lead to eventual grade-level performance. 



New Jersey 

The 1992 report described a new monitoring approach being implemented in New 
Jersey that 'nvolves a visit to every school district each year. This new approach is based on 
the premise that a close working relationship between the SEA and the LEAs eliminates the 
'gotcha' aspect of monitoring. The goal of this new monitoring system is to form a positive 
partnership to improve programs and services for students with disabilities rather than 
concentrate only on non-compliance. A four-year program review cycle is used, with one- 
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quarter of the compliance requirements identified for attention each year. For example, one 
cluster of issues includes district policies and procedures, the utilization of federal funds, and 
due process hearings. During a school year, the state provides intensive training and technical 
assistance in these areas for all districts, and each district completes a self-study. The 
program review during the following school year consists of onsite and offsite activities to 
verify the self-study and to identify effective practices and programs. Corrective action plans 
are drawn up where needed and technical assistance is provided. During the year that all 
districts are reviewed on the first cluster of issues, training and technical assistance are 
provided on the second cluster of issues scheduled to be reviewed in the following year. This 
pattern is repeated throughout the four-year cycle. 

According to New Jersey SEA staff, the new monitoring process has been received 
very positively throughout the state. Districts are making changes to address compliance 
problems before the onsite visit and there have been very few corrective actions prescribed 
as a result of onsite reviews. In fact, there was not one corrective action plan in the first year 
of implementation. The SEA staff also finds the process results in improved relationships 
wiii LEAs and the eliminaes the elaborate corrective action system. 



Texas 

Very little information was included in the 1992 report about Texas because the SEA 
was in the process of redesigning its compliance monitoring system. This design was the 
result of a federal monitoring visit which found Texas in non-compliance. Subsequently, the 
SEA completely revised its monitoring approach that is now known as the Results-Based 
Monitoring System (REM). In addition to ensuring compliance, the new system design has 
two goals: to establish clear linkage between compliance and improved student performance, 
and to create collaborative relationships among SEAs, LEAs and ESCs (educational service 
centers). 

As the first step in the RBM process, the LEA establishes a local review committee 
at both the school and district levels. This committee is composed of individuals from the 
entire range of LEA staff as well as non district individuals such as members of the local 
advisory group or personnel from other LEAs. The committee plans the review based on the 
structure of the special education program. One of the first activities is to gather infoi i/ntion 
from parents. As with most of the review activities under the RBM approach, the method(s) 
to be used to obtain parent input is decided by the LEA. One option is using a representative 
sample of parents equal to the number of student eligibility folders reviewed during the local 
monitoring process. The LEA also determines which student foldej s will be chosen for the 
eligibility review, with the total number based on the average daily attendance of the district. 



State Compliance Monitoring Practices An I 'pilatc 
Project FORI <M at SASDSE 



Page IS 
March J, /W5 



A minimum percentage of student folders based on the incidence levels in each disability 
categories must be included. 

The local monitoring document (Texas Education Agency, 1993) provides extensive 
explanations and forms that cover the indicators to be used. There are 19 Program Excellence 
Indicators to identify practices that result in successful outcomes for all students, and 58 
Compliance Indicators to be reviewed to verify that minimum federal and state requirements 
are met. The form for each excellence indicator poses the issue, lists the sources of 
documentation, and provides a five-point Likert scale for rating the district's current program 
in that specific area. The five criteria range from no evidence to high confirmation of 
implementation of the excellence indicator. Explanatory notes are provided to assist the 
reviewer in understanding the indicator and the scaled criteria. Similar forms are used for 
each of the 58 compliance indicators, except that the decision of the review is limited to a 
yes-no response rather than a rating. 

Under the RBM system, the SEA will cite a discrepancy only when it occurs 
systematically throughout a campus, LEA or cooperative, more closely matching the process 
used in the federal monitoring system. The SEA. will consider a number of factors to 
determine if the violation is systemic or a simple human error. As a general rule, a 
discrepancy will be cited when a violation is found in 30 percent or more of the folders 
reviewed. For violations of a more serious nature such as not conducting an assessment 
before placement, not developing an IEP for continued placement or any violation that 
impacts negatively on the appropriateness of the student's education, a citation will be made 
whenever a single violation occurs even if in only one student folder. LEAs must develop 
action plans for correcting compliance discrepancies and submit them to the SEA for 
approval, but the LEA has maximum flexibility in choosing the strategies to be used. The 
LEA is not required to submit corrective actions for the program excellence indicators. 

Although the RBM indicators are still under review during this pilot implementation, 
the following is a sample of the excellence indicators, grouped under the seven strands, from 
the draft local monitoring document. 

A. Family and Community Involvement: 

• Do families express an above average level of participation as partners in the 
educational process? 

• Does the LEA provide training to empower families to address factors that 
contribute to student success? 

B. Student Eligibility: 

• Is a series of intervention strategies designed by qualified individuals tried prior 
to a referral for special education? 
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• Do assessment procedures provide information that allow goals to be developed 
that focus on student strengths? 

C. Student Access: 

0 Do eligible students participate in all statewide mandated assessment measures 
as appropriate? 

° Does the LEA administer alternative performance measures for students who are 
exempt form state mandated achievement tests? 

D. Implementation: 

• Do students who need modified/adapted texts and other materials receive them 
at the same time as their nondisabled peers? 

• Are students with disabilities provided instruction in chronologically age- 
appropriate educational environments? 

E. Transition: 

« Are students with disabilities successful in transition to the next appropriate 
educational setting or to the community? 

F. Support: 

• Do districtwide committees seek input from and collaborate with representatives 
from the special education staff? 

° Does the LEA use teacher/student support teams to ensure student success? 

G. Program Evaluation: 

« Has the LEA developed and implemented formal evaluation systems that use a 
variety of strategies to measure effectiveness? 

The RBM system is now in its second year of pilot testing. The Texas SEA sees the 
it as a true shift in priorities moving away from a total process orientation to a focus on 
program excellence without neglecting compliance indicators. 

DISCUSSION AND CONCLUSIONS 

The results of the 1994 survey combined with information gathered from the Third 
National Monitoring Conference confirm that problems with compliance monitoring discussed 
in the 1992 report have not improved. Although a few states are trying to shift their 
monitoring emphasis away from a total concentration on input and process variables in order 
to incorporate outcomes variables, these efforts are too few to be considered representative 
and too recent to be evaluated as to their eventual impact on all states. 

As noted in the 1992 report, federal standards for compliance are still not specified. 
OSEP has draft standards for monitoring that have been used as a guide for conducting pre- 
site and onsite activities relative to compliance. However, t' e word "standards" can be used 
in two ways in reference to monitoring: a) monitoring standards are the areas of state 
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programs to be reviewed and the documentation required for demonstrating compliance; and, 
2) compliance standards are the specific levels that constitute acceptable conformity for each 
monitoring standard. For example, the IEP is used as evidence of a number of requirements 
including the provision of a free appropriate education (FAPE) for students with disabilities. 
What is the com pliance standard that should be applied in judging the state's compliance with 
IEP requirements? If one error is identified on an IEP such as a missing parent signature, 
is that sufficient evidence to cite the state for non-compliance? OSEP advises that its 
intention is to focus is on systemic problems, but there are no consistently applied guidelines 
for making a finding in a specific area. 

An analysis of compliance monitoring practices at the federal and state levels in the 
context of education for the 1990s and beyond indicates a need for a total re-examination of 
the monitoring system. The process has not changed significantly from the original 
monitoring procedures described in a memorandum entitled Program Administrative Review 
Process: An Overview issued in 1977 by the Bureau of Education for the Handicapped, the 
federal agency now known as the Office of Special Education Programs (National Association 
of State Directors of Special Education, 1977, p. 222-30). The many and varied changes 
being adopted as a result of the educational reform and inclusion movements should provoke 
a revision in the approach to monitoring and its position in the overall goal of improving 
education for students with disabilities. 
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PROFILES OF STATE MONITORING SYSTEMS 
REGIONAL RESOURCE CENTER PROGRAMS 



Purpose: 

This information is being collected to prepare a resource document for use by SEA staff responsible 
for developing and improving SEA program monitoring systems for infants, children and youths 
with disabilities. The 30-40 page product will include a matrix that serves as a quick index to the 
characteristics of state systems plus some contact and narrative information. The document will be 
distributed to all 60 states and jurisdictions currently receiving Federal Special Education funds and 
to other interested organizations and agencies. 

Directions: 

1. Please complete this form and add a brief narrative. 

2. Your answers should describe your system as of March 15, 1994. If you plan to make 
changes, indicate so in item #40 and in the brief narrative. 

3. Send or call in your responses to your Regional Resource Center contact person by April 
11, 1994. 



Contact Information: 

Current Date: : 

State: 

Name of Primary Contact for Monitoring: 

SEA Unit: 

Address: 

City/ State/ Zip: 

Phone#: Fax#: 

SpecialNet: Internet: 

Demographics: 

1 Number of entities that must be monitored: 

a)LEAs b) lEUs c) Other State Entities d) Private Entities 
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2a IDEA child count taken December 1, 1993 for ages 3-21: 

2b Chapter I child count taken December 1, 1993: 

2c Total Child Count (IDEA + Chapter I): 

3 Name of agency that monitors Part H: 



Staffing: 

Full Time Equivalent (FTE) staff employed for monitoring (Note: 1.0 FTE might represent several people 
art time) 



4a 
4b 



Located in SEA Central Office 
Located elsewhere: 



FTE 



(FTE) 



5 Changes in number of monitoring staff (+ or - FTE) since March 1, 1993: 



Personnel used as part of monitoring team(s) include: 

6 SEA staff (regardless of office location) 

7 LEA peers 

8 Parents 

9 Others (e.g., IHE faculty, contracted personnel) 
(If yes, please specify in state narrative) 



Y 
Y 
N 
Y 



N 
N 

N 



Process and Materials 

10 Our current cycle (in years) for comprehensive onsite visits is 

1 1 Child count verification is part of our onsite process. 

12 We use focused/targeted monitoring. 

(If yes, briefly describe in state narrative) 

13 LEA self-monitoring is a part of our process. 

14 Our monitoring is conducted in conjunction with other SEA units 

15 Our monitoring is conducted in conjunction with the Part H Lead Agency 

16 Our monitoring goes beyond legal obligations to include program 
quality/effectiveness. 



years. 



Y 
Y 

Y 
Y 
Y 
Y 



N 
N 

N 
N 
N 
N 
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17 Our monitoring goes beyond legal obligations to include student 
outcomes/results. 

18 We provide TA as a formal part of our monitoring process. 
(If yes, briefly describe in state narrative) 

19 We formally collect information fix>m parents (e.g., via surveys, 
interviews, public forums). 

20 We use a computer database of LEA data of off-site monitoring. 

21 We use computers during on-site process:. 

22 We generate our reports via computer (other than word processor). 

23 LEA application review is a part of our monitoring process. 

24 LEA policy/procedures review is a part of our monitoring process. 

25 LEA interagency agreements review is a part of our monitoring process. 

26 We use sanctions other than fiscal sanctions. 

27 We have written interpretative standards for assessing compliance. 

28 We have written criteria for selecting LEAs to be monitored. 

29 W have an instruction/training manual for monitors. 

30 We use a consistent sampling formula for selecting student records. 



Forms 

3 1 We have standardized forms available for LEA adoption. 

32 We have and use an LEA policy and procedures review form. 

33 Our IEP record review form goes beyond compliance issues. 

34 We have and use interview forms designed specifically for monitoring. 

35 We have and use mail or telephone survey forms. 

36 We have and use a summary checklist (to integrate findings). 

37 We have and use a standardized monitoring report/narrative format. 



ERIC 



107 



Changes 

38 We have made adaptations in our monitoring system to reflect LEA/ 
Head Start collaboration. 

39 We have made adaptations in our monitoring process to address unique 
early childhood issue* (Section 619/preschool, Part H to Part B transition) 

40 We are making or considering major revisions to our system. 
Narrative Summary 

Please provide a summary of the key elements of your system, including information related to items (9, 
12, 18) limiting yourself to approximately 300 words (attach an abstract if one has already been written 
and use extra pages if necessary). Please cover the following in your narrative: 1) Overview of the system 
(general procedures/approach). 2) Features that might be considered unique. 3) Changes being considered. 

Please indicate materials on the following topics that you are willing to share on request: 

a Materials that might help others improve systems of corrective action and technical assistance. 

b Materials that might heip others to use unique or "state of the art" techniques in monitoring (e.g., 

technology, linking cyclical with focused monitoring , databases). 

c Materials that might help others to monitor the IEP. 

d Materials that might help others to develop interpretive standards, define required documentation 

and/or maintain consiste ncy among monitors. 

e Materials that might help others with the issue of "monitoring and the school reform movement." 

f Materials that might help other unitary SEAs (SEAs that are also LEAs) to conduct monitoring. 

g Materials that might help others in monitoring for outcomes. 

h Sample copies of technical assistance documents. (Please specify): 

i. Other (please specify): 

Please remember: 

Send or call in your responses to your Regional Resource Center contact person by April 11, 1994. 



Y N 

Y N 

Y N 
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APPENDIX B: Chart of Selected Survey Responses 



U 



9 
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Notes on the chart contained in Appendix B: 



Some of the data contained in the original report of the RRC-conducted survey were 
discussed with SEA personnel and some revisions and additions were made available. The 
asterisks on thf; chart are explained as follows: 

* Hawaii is a one-district state, so the number of LEAs is not applicable. 

** In Michigan, intermediate units monitor their LEAs during the two years that the 
SEA does not monitor them. Each EEU has part-time staff assigned to this task 
approximately half time. 

*** Massachusetts has 22 staff whose responsibility includes monitoring all program 
areas including special education, complaint management and technical assistance. 

**** j n ^f ew j erse y 5 every LEA is monitored every year on approximately one- 
quarter of the total requirements. This system is discussed more fully in the 
analysis section of the report. 

It should also be noted that, because of time constraints and the difficulty in 
establishing contact to verify and discuss their responses, data for the 10 non-state 
jurisdictions was not included in the in the analysis narrative. The chart does contain the 
responses submitted by the nine jurisdictions that responded to the survey for the specific 
components covered in this report. 
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A FRAMEWORK FOR ACCOUNTABILITY: 
Concepts, Approaches, and Issues 

By Edward J. McCaul, Ed.D. 



Introduction 

During the latter part of the 1980s, public education has been the focus of considerable 
attention and concern. Since "A Nation At Risk" was published in 1983, many State-level efforts 
to reform and improve public education were initiated, and the Federal government proposed its 
"America 2000" strategy with six national education goals. With the promulgation of these goals, 
a clear need arose for measuring student progress and performance. In addition, international 
achievement tests revealed an increasing achievement gap between U.S. students and those of 
other lands. 

These developments have led to a growing emphasis on the results of the educational 
process and the student outcomes that the process yields. In recognition of this new concern with 
student outcomes, the Office of Special Education Programs (OSEP) established the National 
Center on Educational Outcomes (NCEO) at the University of Minnesota. In a subcontract with 
NCEO, the National Association of State Directors of Special Education (NASDSE) seeks to 
provide support for the work of NCEO, particularly in the areas of accountability and State-level 
policy, and to collaborate with State Directors of Special Education in identifying and analyzing 
critical issues regarding educational outcomes for students with disabilities. 

Discussions with State Directors have indicated that the issues of special education 
accountability are becoming increasingly important. Along with the focus on outcomes, the idea 
of holding educators accountable for results has gained support (Clark & Astuto, 1989; Kirst, 
1990). Many States are struggling with fiscal crises that involve tough decisions about the 
allocation of resources. Special education budgets are particularly vulnerable to attack given the 
rising costs of special education and a resulting "backlash" of public opinion against special 
education (Zirkel, 1990). Other concerns include the growth of students identified as learning 
disabled, the increasing concern over what constitutes an "appropriate" related service, and some 
evidence that suggests many students with disabilities experience poor educational outcomes. 
Given these circumstances, accountability regarding special education programs and services 
becomes vitally important and special education policymakers are asking questions such as: How 
do we hold local programs accountable for results? What are the basic elements of an 
accountability system? What types of systems are States currently using to ensure accountability? 

The Outcomes Project at NASDSE. convened a planning meeting during the summer of 
1992 to design activities to address issues regarding the development of fair and effective State- 
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level .accountability systems. Participants included representatives of NCEO, the Center for 
Policy Options, the Office of Special Education Programs (OSEP), and a State Director of 
Special Education. One component of the workplan developed at this meeting culminated in the 
development of this report. 

This report provides an overview of issues regarding the development of State-level 
systems of accountability, a description of some core elements for accountability systems, and 
an examination of the accountability systems in selected States. Specifically, it is divided into 
the following sections: (1) a working definition, (2) a review of the concepts of accountability 
provided in the literature, (3) a discussion of factors involved in designing accountability systems, 
(4) a brief overview of issues regarding special education, (5) a description of accountability 
systems in selected States, and (6) a conclusion and recommendations. 



Definition 

As defined by Webster, to be accountable is to be "subject to giving an account: 
answerable." This seemingly simple definition belies the substantial, and sometimes ominous, 
overtones that the term "accountability" has taken on for the educational community. 

The idea of holding educators answerable for results is not a new one. As early as the 
beginning of the century when the factory model of education was in vogue, school reformers 
discussed the need for schools to turn the "raw material" of students into the "end products" of 
educated citizens. In 1916, Cubberly stated: 

Our schools are, in a sense, factories in which the raw products (children) are to 
be shaped and fashioned into products to met the various demands of life. The 
specifications for manufacturing come from the demands of the twentieth-century 
civilization, and it is the business of the school to build its pupils according to the 
specifications laid down. This demands good tools, specialized machinery, 
continuous measurement of production to see if its is according to specifications, 
the elimination of waste in manufacture, and a large variety in the output (cited 
in Hanson, 1985, p. 24). 

Concepts of accountability have evolved substantially since Cubberly's era. According 
to Hanson (1985), the 1970s ushered in a new era of "accountability." However, as a result of 
the publication of A Nation At Risk during the 1950s and subsequent reports about the status of 
American public education, issues of accountability have emerged as major themes for public 
education in the 1990s (Kirst, 1990; Rebarber, 1991). The section below discusses some of the 
different concepts of accountability presented in the literature. 
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Concepts of Accountability 



In 1974, Henry Levin provided an overview and comprehensive discussion of issues 
related to accountability in education, and he anticipated many of the current concerns over 
educational productivity and student outcomes. Levin offered one of the earliest conceptual 
frameworks of accountability; specifically, he viewed accountability as assuming one or several 
of the following different forms: 

• As performance reporting in which periodic reports are issued over the 
attainments of schooling. These reports may be based on financial or student 
performance results or both. An assumption is that by providing performance 
information, accountability will be enhanced. 

As a technical process in which the problem is viewed as "delivering the goods 
at a reasonable cost." This approach emphasizes standardization, contracting for 
performance, and elimination of unproductive staff (usually teachers or local 
administrators). It may involve a "cost-benefit analysis" of educational services. 

As a political process in which schools are seen as the result of what a particular 
constituency or constituencies desire. Various coalitions compete for control over 
the school processes and outcomes, and the most powerful constituency holds 
sway. 

As an institutional process in which the current "class structure" becomes 
reproduced. Advocates of this viewpoint often support "deschooling" through the 
abolishment of compulsory attendance and a major overhaul of our society and 
schools. 

In Levin's view, true educational accountability is difficult, if not impossible, to achieve 
due to conflicting political pressures on schools and the power of the "educational professionals 
themselves" in the process of shaping public schools. Further, to be truly accountable, schools 
would need a consensus on the educational outcomes of schooling. Levin views outcomes as 
falling into two areas: (1) proximate, the immediate and tangible ,.«ults of schooling such as 
enhanced achievement test scores; and (2) ultimate in terms of -~ social benefits and costs to 
society. Levin argues that there is a lack of clear consensus on these outcomes and further that 
"we're on exceedingly shaky ground when we try to translate educational outcomes into social 
outcomes since the latter are generally considerably removed in time and space from the former, 
and a dynamic social, political, and economic structure is likely to alter these relationships over 
time" (P. 387). And, of course, educators have articulated the objectives of schooling as 
enhancing these ultimate goals. 
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In delineating these complex relationships among political processes, educational inputs, 
and educational outcomes (both proximate and ultimate), Levin concluded that: 

Perhaps the most important implication of this review is to suggest that a 
significant tightening of the accountability linkages in education is probably 
impossible without substantial changes in the governing processes and organization 
of the educational sector. Yet educators are fond of talking about accountability 
as a technical problem which does not require any major restructuring of 
institutions. Such a viewpoint may place the educator at center stage in the 
accountability movement, but it is not likely to make much of a difference in the 
overall functioning of the schools or society (p. 388). 

While Levin's work has provided a foundation for other discussions of accountability 
(e.g., Kirst, 1990), other frameworks for accountability have been offered. Darling-Hammond 
and Ascher (1991) addressed issues of accountability facing "big city schools", and they argued 
that there are at least five types of accountability systems being used: 

Political accountability through the election of legislators and school boards. 

Legal accountability through cases involving the accountability of school systems for 
desegregation or provision of equal opportunity. 

Bureaucratic accountability through State and district education offices enacting and 
monitoring compliance with rules and regulations. 

Professional accountability through teachers and other school staff acquiring professional 
knowledge and adhering to professional standards of practice. 

Market accountability through parents and students choosing the schools that they believe 
are effective and meet their particular needs (p. 3). 

The authors argue that the two most prevalent forms are legal and bureaucratic accountability, 
but that these two forms may have "overextended their reach" and become less useful. They 
stated that "if school indicators are improperly designed or unintelligently used, they can actually 
undermine accountability" (p. 2). 

Kirst (1990) also described a framework for accountability. He argued that six broad 
approaches to accountability that were being used in education: 

• Monitoring may be used to assess levels of compliance with standards or 
regulations. 
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Reports of student performance (e.g., district and school State achievement test 
results) may be published. 

o Systems that provide incentives may be developed. 

A "free market" approach may be used to provide accountability. 

Changing the locus of authority or control of schools may be used to designate 
ownership and responsibility for the achievement of results. 

« Changes in professional roles may help to enhance accountability (p. 1). 

As Kirst described the approaches listed above, he noted a reliance on business as the 
model for educators to emulate, on "objective measures" to evaluate student performance, and on 
accounting procedures and cost control as methods for improving education (p. 4). Clark and 
Astuto (1989) argued that accountability procedures such as those listed above are consistent with 
the Federal policy on education through the years of the Reagan and Bush 
administrations. These authors also argued that the public favors businesslike "quality 
control" types of accountability approaches such as a standard nationwide exam to obtain a high 
school diploma, a State board examination for teachers, and the raising of academic standards (p. 
16). According to Clark and Astuto, the public favors greater competition and believes that 
State-by-State comparisons would serve as incentives to local schools for improving their 
performance. They also believe that the public favors parental choice programs (p. 17). 

Approaches to educational accountability that emphasize the financial "bottom line" have 
traditionally been viewed with suspicion by many special educators. The origins of current 
special education laws are rife with stories of children with disabilities being excluded from 
public schools because of the "excessive cost" of their education 1 . Attempts to model schools 
after businesses and to make them more "cost effective" may be viewed by special educators and 
advocates as threatening the provision of high cost services and accommodations necessary for 
children with disabilities to have equal opportunity to benefit from public education. As 
discussed later, however, there have been some attempts to conduct "cost-benefit" analyses of 
special education services. 



o 



1 The testimony presented before Congress during the hearings on P.L. 94-142 contain numerous stories to this 
effect. (See, for example, Alexander & Alexander, 1980) 
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Factors in Designing Accountability Systems 

Based on a review of the literature, several factors emerge relative to developing a 
framework for viewing accountability. 

Level of accountability. Issues of accountability are relevant at the Federal, State, and 
local level. While this seems straightforward, the complex interrelationship among these levels 
of accountability may be difficult to untangle. For example, the manner in which a State chooses 
to hold locals accountable for Chapter II programs will most likely be dictated to a large degree 
by the program's Federal regulations. The intent of this paper is to focus on State-level 
accountability and the State's position relative to the Federal and local levels. 

Who will be held accountable and for what. Since the focus of this paper, and the 
conceptual framework for accountability, is at the State level, to some degree this question is 
answered — the State will hold the locals accountable for results. Nevertheless, there is 
considerable room for interpretation over who at the local level will be accountable for what 
results. Many parties share responsibility for achieving educational outcomes: students, 
paraprofessionals, teachers, building and district aciministrators, the school committee, and the 
school community. Accountability systems need to be sensitive to these multiple roles and 
responsibilities. 

Difficulties in analyzing alternatives. In developing accountability systems, State-level 
policymakers need to grapple with several complex and difficult issues. Analyzing the efficiency 
of various approaches or conducting a "benefit-cost analysis" of different strategies is not as 
straight forward as it is in the private sector. 

For example, Monk (1992) examined the nature of traditional "production function" 
research that sought to discover systems of accountability and cost-efficiency by exarnining the 
relationship of various "inputs" (e.g., expenditures, teacher-student ratios, etc.) to various 
educational outcomes, usually student achievement or labor market success. After reviewing the 
production-function literature, Monk concluded: 

Numerous states and some school districts have implemented reforms containing 
outcome-based incentives... In so doing, the more centralized authority sidesteps 
having to spell out the ingredients of education success and can sit back and act 
as judge and jury of those with the more immediate responsibility of producing 
the desired results. This policy response can be viewed as a strategy , perhaps 
even an ingenious strategy, that successfully finesses the ignorance that 
characterizes our knowledge of the underlying education production function(s). 
Ingenious though this "outcomes as standards" response may be, there are serious 
deficiencies that are not sufficiently well appreciated (pp. 307-308). 
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Relationships among stakeholders. Ferris (1992) argued that contractual obligations are 
impticit in many of these nested levels in our educational system and used the "principal-agent 
framework" to analyze educational accountability. This framework has traditionally focused on 
contractual arrangements in the private sector but Ferris found it useful for analyzing public 
sector endeavors such as education. In the context of this framework, the term "principal" does 
not necessarily refer to school-based adrninistrators, but to a person or party contracting with an 
"agent" for delivery of services, in education, citizens enter into an implied "contract" with 
Federal and particularly State governments for the delivery of educational services. The States, 
in turn, enter into arrangements with local school districts and within the district the school board 
contracts with school district staff to implement educational policies. Further, the district may 
choose to delegate authority to the school level and hold the school accountable for results. 

This school-based decisionmaking approach has been increasing in popularity, but it is not 
without potential problems. The "agent" carrying out the implied contract may have different 
interests and objectives than the party contracting for services and may pursue these objectives 
at the expense of the goals of the contracting party. Holding the agent accountable may be 
difficult due to the problems associated with accurately measuring the desired outcomes, and 
enforcing the "contract" may be costly. These problems apply to districts initiating site-based 
decisionmaking approaches but they are equally applicable to State-level accountability systems. 
Ferris (1992) has stated that the devolution of key decisions to the school site developed out of 
a desire to capitalize on the school's superior knowledge about the school's clientele and 
immediate educational environment. However, many recent State-level reforms (e.g., Statewide 
testing and curricular goals) have a centralizing tendency and may erode local-level autonomy. 



Need for flexibility. Mohrman, Lawler, and Mohrman (1992) also discussed issues facing 
schools in considering such contractual arrangements and in instituting systems of participatory 
decisionmaking. These authors argued that the educational process is "complex, uncertain, and 
highly interdependent," and thus should involve high levels of teacher involvement in 
decisionmaking. This approach involves "moving the design process as close as possible to a 
particular customer base, thereby reducing the number of political agendas that have to be 
accommodated" (p. 359). The authors view this process as complex and "no one way fits all": 

In this process, the multiple stakeholders get together to design their organizational 
unit. They first clarify the values that they are trying to optimize, educate 
themselves about their choices and the irade-offs they must make, and determine 
how their future must differ from the status quo. Only then do they determine a 
design, implement it, and set up a process to leam how it works and to tinker or 
change it until they achieve their desired results. A self-design change process is 
a participative learning process for guiding fundamental organizational change (p. 
358). 
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In other words, there is no clearly defined blueprint for the success. Accountability systems must 
be flexible and open to revision based on results and on feedback from educators involved in the 
process. 



Special Education and Accountability 

Attempts to assure accountability in special education are built into Federal and State 
statutes and regulations. The major approach to special education accountability has been 
monitoring for compliance with these statutes and regulations. Monitoring takes place on two 
levels: The Federal government monitors States and the States monitor local school districts. In 
both cases, the monitoring process that has evolved has become quite elaborate, intensive, and 
"highly visible" with considerable energy being expended in preparing for monitoring visits and 
responding to issues identified in the monitoring process. At the same time, monitoring has 
focused on regulatory and procedural compliance rather than on results (Rostetter, 1988). 

The monitoring process has been focused upon ensuring that students with disabilities have 
access to education and that certain prescribed procedures are followed (Ahearn, 1992). The 
monitoring process does not, therefore, ensure the attainment of specific results. The presence 
of certain procedural requirements, such as the annual review of an IEP, do imply at least some 
level of accountability - the school must convene a meeting to discuss the student's program and 
the possibility exists that the parent may hold the school accountable for changing a student's 
program if it is deemed inadequate. Similarly, implicit in requirements such as a parent's right 
to an independent evaluation is a system of "checks and balances" in which parents may hold the 
school to some level of accountability. 

At the same time that monitoring has evolved into such a formal and complex process, 
other perhaps less comprehensive methods of assessing accountability in special education have 
been attempted. In fact, the IDEA requires State plans to include provisions for an annual 
evaluation of the effectiveness of programs in meeting the education needs of students with 
disabilities (Gonzalez, 1992). However, in spite of this requirement, the results of a recent study 
indicated that "the majority of States continue to use compliance monitoring procedures as a 
program evaluation tool to a significant degree, and findings from monitoring activities, 
presumably, as evidence of the effectiveness of special education programs and services" 
(Gonzalez, 1992, p. 10). 

The research community has been actively involved in evaluating special education 
programs, and evaluation components are required in program model demonstration grants from 
the Office of Special Education Programs (DeStefano, 1992). On a national level, studies of 
youth in transition, such as the National Longitudinal Transition Study (NLTS), have examined 
the relationship of special education services and the postschool experiences of youth with 
disabilities. The results provide some measure of "what works" and therefore standards for 
accountability. In a somewhat related effort, Lewis et al. (1988) conducted a cost-benefit analysis 
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of special education and argued that this process provided some measure of accountability: "The 
principal benefit to special education is the focus on the usage of resources in relationship to 
outcomes. Such analyses force administrators and policy makers to address questions of resource 
usage in relation to expected postschool benefits for students with handicaps [sic]" (p. 212). 

Finally, special educators have used the "changing of professional roles" cited by Kirst 
as measure of accountability. Personnel preparation has always been a crucial element of special 
education, and it has been argued that special educaars already have many of the skills that are 
being sought for tomorrow's general education teachers (Cook & Friend, 1991; Sailor, 1991). 
In addition, models of personnel preparation are changing in response, in part, to school reform 
and more collaborative teaching models are being explored (Friend & Cook, 1992). By 
enhancing professional status, roles, and responsibilities, important element of accountability may 
be incorporated into special education programs and services. 



Common Elements in State-Level Accountability Systems 

As is evident from the discussion above, several types of accountability systems exist. 
This section presents a framework for analyzing the core elements in any accountability system. 

Content. The question here is for what are designated parties to be held responsible? 
For lowering absentee or dropout rates? For raising student achievement on one or more 
achievement tests? What about objectives regarding affective or ethical development? What are 
the specific indicators that will demonstrate achievement of the chosen outcomes? 

Discussions over the desired outcomes of education are relevant. The problems of clearly 
articulating the outcomes of education have long been noted (for example, see Hanson, 1987). 
The National Center for Educational Outcomes (NCEO) recently convened a series of meetings 
during which various stakeholders generated lists of desired educational outcomes. The result 
was a list of some 200 outcomes! (Ysseldyke, personal communication, June 1992). For obvious 
reasons, it is essential to be clear over uie content of accountability, and careful consideration 
needs to be given to this determination. 

Measurement. Once the questions of content for accountability is determined, how will 
we accurately measure the indicators of achievement of these chosen outcomes? In some cases, 
the answer appears obvious because what is being measured is in quantifiable form - for example, 
student achievement tests. Questions of the technical adequacy of assessments raises a host of 
complex questions, however, and these become even stickier when we strive to make comparisons 
and fair decisions over how to administer accountability systems. Should we hold everyone to 
certain standards or do we make adjustment for background factors (e.g., district socioeconomic 
status) in making comparisons and administering consequences? How do we know that we have 
accurately measured this factor? What about other factors that prevent a "level playing field"? 
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Measurement presents a different type of problem with less quantifiable educational 
results. Space here does not permit a review of the "quantitative v. qualitative debate" that has 
enlivened the pages of many educational journals for at least the last 15 years and arguably much 
longer 2 . It is clear, however, that certain results of education present particularly difficult 
measurement challenges. How does one measure "citizenship"? How does one measure the 
types of social and interpersonal skills that young people will need to compete in tomorrow's 
work force? The argument presented here is not that these are inherently non-quantifiable but 
that these types of skills may be important results of public education, and it may be quite 
difficult to measure these, to make fair comparisons, and to hold educators accountable for 
results. Again, States need to consider carefully the available options regarding measurement 
issues in developing their overall accountability system. As stated by Monk (1992), "incentives 
can be well or poorly designed. They can give rise to perverse and unexpected effects" (p. 316). 

Reporting and use of data. Once decisions have been made regarding content and 
measurement issues, policymakers must decide how to use the results and the best methods and 
formats for reporting them. The two decisions are intertwined and choices over format are 
related to decisions over the target audience for the report. Lengthy reports are not likely to be 
successful in mobilizing the public's support; face-to-face contact with key political players may 
be unlikely. States have experimented with "report card" types of approaches to providing 
considerable information about a district' s performance in a quickly digestible fashion. As noted 
above, however, such an approach may lead to a narrow focus of desirable outcomes or foster 
a negative school climate. 

Consequences. Another important decision in designing an accountability system involves 
deciding on how to reward good performance and to punish or ameliorate poor performance. 
Rewards may be fiscal or they may involve good public relations (e.g., the publishing of 
Statewide test results in newspapers), or other benefits or privileges (e.g., the "waiver" of certain 
State requirements. Sanctions may take the form of increased State intervention, fiscal penalties, 
public exposure (again, the publishing of test results is an example), or other forms of negative 
public relations. It is worth noting that some forms of sanctions and rewards may have an effect 
on morale, school climate, and community-school relations. 

All the elements cited above are intertwined, and decisions over one element affect 
decisions in the others. It is also worth noting that issues concerning staffing and financial 
resources cut across all of the above elements of accountability. For example, reporting results 
in a variety of formats is clearly desirable but not necessarily cost-effective. 



2 Gage (1989) provides a review of these debates in his article, "The paradigm wars and their aftermath: A 
'historical' sketch of research on teaching since 1989", in the Educational Researcher, /<S(7), 4-10. 
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External, and to a large degree uncontrollable, factors such as the State's budget affect 
the design of an accountability system. The design may depend on how the State is being held 
accountable by the Federal government or the citizenry at large. As an example, pressure from 
the public to demonstrate increased student achievement may substantially affect the ways in 
which the State holds local districts accountable for results on Statewide assessments. 



State-Level Accountability Systems: Alternative Approaches 

One of the consistent themes in the literature on educational accountability is that there 
is no one "best way." Difficulties arise due to the multiple stakeholders involved in the 
educational process and the uncertainty over the specific organizational and instructional strategies 
that are sufficient conditions for achieving desired results. As stated by Mohrman, Lawler, and 
Mohrman (1992): "The political nature of schools makes the transition [to organizational change 
and enhanced educational performance] more difficult than in private-sector firms. Agreement 
on desired outcomes and on assessment of current performance is hard to come by, and lack of 
agreement is likely to paralyze needed reform" (p. 359). 

Given the lack of one clear and best approach, each State's choice over methods of 
ensuring accountability needs to be sensitive to the complex and unique relationship in each State 
among the citizenry, the State Board of Education, the State Education Agency, the local districts, 
and the local schools. The accountability system must be designed to match the political and 
demographic conditions to which it will be applied. Decisions about desired results, degrees of 
local autonomy, and the nature of contractual arrangements (whether explicit or implied) for 
producing desired results must all be carefully tailored to the particular circumstances of the 
State. It is in this context that each SEA needs to consider the content, measurement, reporting 
and use of data as well as the consequences for performance that will form the basis for their 
accountability system. 

Because this process of developing, assessing, and constantly revising accountability 
systems should be a dynamic and ongoing process, States may benefit from the experience of 
other States in developing and testing alternative models of accountability. Therefore, the 
sections that follow highlight the accountability systems that have been developed in three States. 
While these States have given careful and considerable thought to designing and implementing 
accountability systems, they are not intended as "exemplars" that all other States should follow. 
Rather, they are offered as examples of different approaches that States have taken in designing 
their accountability systems. The accountability systems of New Jersey, Virginia, Michigan and 
West Virginia are described below. 
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New Jersey 



One of the outcomes of public education is the high school diploma, and New Jersey 
closely examined their diploma issuance system in 1987. As a result, New Jersey has a single 
diploma system that provides for granting a diploma on the basis of attendance, course credits, 
local requirements, and a measure of high school proficiency. The high school proficiency test 
measures achievement at the 9* grade level, but is now shifting to an 1 1* grade standard. 

A proposal was made to establish a separate diploma for students with disabilities when 
the Statewide diploma was being considered in 1987. The Department of Education opposed this 
provision and developed a proposal that would establish the same requirements for all students. 
Students with disabilities would be treated exactly like all students unless, because of the nature 
of the disability, graduation requirements would need to be modified. The State now requires 
all students to pass the proficiency test in order to receive a State-endorsed diploma unless the 
student is exempted in the Individual Education Plan (IEP). Exemptions are granted to students 
whose educational plan does not include proficiencies that are tested, but an exempted student 
must demonstrate proficiency in a set of alternative areas outlined in the IEP. 

As an example, a student with a moderate to severe cognitive disability might not be 
taught the math proficiencies addressed in the Statewide assessment. Therefore, the student could 
be exempted from those proficiencies and would have to achieve alternative proficiencies as 
proposed in the IEP, e.g., functional academics related to math, usually in the form of a portfolio 
demonstration of proficiency. On the other hand, a student with a learning disability who 
received instructional support in the resource room for the regular curriculum content would have 
to demonstrate proficiency on the Statewide exam. Test accommodations are available for 
students with disabilities such as taking the test in Braille, extra time allowances, etc. 

Test scores are made public in New Jersey, and public pressure for accountability has 
grown for local districts to increase the number of students passing the proficiency exam. Many 
educators in New Jersey speculated that there would be pressure to identify more students as 
eligible for special education and therefore eligible to take the exam with modifications. Contrary 
to the dire predictions, more students took the Statewide test and the number of students passing 
the exam also increased. This may change, however, when the new 11 th grade standard of 
proficiency for passing is adopted. The graduating class of 1995 will be the first class tested at 
the 11 th grade proficiency level, and this may lead to an increase in the number of special 
education students being exempted f 4 om Statewide testing. 

In another effort to examine and enhance the accountability of local districts, the State 
Department of Education gave grants to nine local districts to examine three issues: post-school 
employment for students with disabilities, special education dropouts, and special education 
students' achievement on standardized achievement tests. These grants were initiated eight years 
ago and ended after four years. The results and successful practices of these projects were 
published in a report on the transition of secondary special education students. 
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Since then, the New Jersey Department of Education has been developing plans to move 
toward a more outcome-oriented approach to special education. Currently, monitoring is 
predominately focused on compliance, but there is an attempt to incorporate outcomes in the 
process. The State is also looking to develop self-study guides and has attempted to emphasize 
a self-study, self-evaluation type of approach to outcomes rather than imposing mandates on local 
districts. This whole effort is in the planning stage and is intended to be flexible and adopt a 
collaborative approach with local districts to enhance accountability for positive outcomes. 



Virginia 

Special education accountability in Virginia is seen as a coordinated effort with general 
education. The demands for accountability by special education are somewhat different from 
those of general education because of Federal special education statutes and regulations. The 
State special education regulations require that local districts submit a plan that outlines all of 
their special education procedures. The LEAs are not asked to report on progress in Individual 
Education Plan (IEP) goal attainment, and there are no formal reporting or program evaluation 
requirements beyond that required by Federal mandates. Most of the data collected from the 
localities is for federal reporting purposes. Data from special education is included in the annual 
Superintendent's report, but this tends to be anecdotal rather than statistically oriented. The State 
does not attach specific sanctions (nor do they offer fiscal rewards) to districts beyond the threat 
of withholding allocations if the district is in regulatory noncompliance. This has not occurred 
frequently, but there has been at least one instance of the State withholding funds because of a 
lack of general education compliance. So far, Virginia has not withheld funds due to a lack of 
special education compliance. 

The Virginia Department of Education has also recently modified its initiative on 
Standards of Quality, which is the guiding document for education in the State, and the special 
education division is seeking to coordinate its accountability efforts with this general education 
initiative and the overall goal of developing a "World Class System of Education" program by 
the year 2000. This plan for a world class system includes four components: 

» All graduates will be prepared to directly enter and continue in the skilled work 
force, or to enter and complete further academic and technical education. 

Establishment of a Common Core of Learning that defines what all students 
should know and be able to do when they graduate. 

The World Class Education Program will measure student mastery, and hold 
schools accountable for the results. 
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Ensuring that all students have an opportunity to master the Comrr.t i Core of 
Learning means trying innovative approaches to instruction and ways of organizing 
schools. 

As with other States, it is expected that Virginia's accountability measures will apply to 
students with disabilities, but the logistics of how they will apply is yet to be precisely 
determined. The Lead Specialist for Special Education serves as the liaison between the special 
and general education agencies, coordinating activities so that the LEAs will act in concert. The 
Virginia Department of Education is examining an outcomes-based approach to accountability and 
is gathering information on outcomes based education and on developing a common core 
curriculum. 

Since 1991, performance indicators on special education have been reported annually 
through Virginia's Outcome Accountability Project. The indicators focus on student outcomes 
and are reported as percentages at the district level. The include: attendance; dropout rate; 
receiving the regular or advanced studies diploma; pass rate on the State's literacy test for 6* 
graders; participation in structured work programs; and, co-curricular participation with non- 
disabled peers. Current emphasis is placed on using the indicator information to improve local 
special education programs and to increase student achievement. 

The Virginia Department of Education operates from the perspective of forming a 
partnership with LEAs for improvement of programs. There is flexibility in placing 
accountability demands on LEAs, and they acknowledge local conditions and respect local 
decision making unless there are clear indications that local decisions are not in the best interests 
of the students. The Department of Education prefers to offer leadership rather than issue 
mandates for the local districts. 



Michigan 

In Michigan, accountability is viewed in terms of forming a partnership with local districts 
rather than taking a prescriptive "top-down" approach. For example, an outcomes approach to 
special education is promoted but not mandated and, in fact, it is not known for sure how many 
districts are using the State guidance in outcomes. 

Michigan has a long history of local autonomy and State policymakers have chosen to 
work with this trend rather than against it. Currently, monitoring is for compliance only and is 
not "outcome-oriented," although current plans include a gradual switch to a technical assistance 
orientation to monitoring and some possible approaches are being pilot tested. 

The Michigan Education Reform Act (PA25) includes components designed to enhance 
local accountability. The Reform Act covers four areas: 
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» accreditation 

• core curriculum 

• annual reporting to citizens 
o school improvement teams 

SEA personnel report that the accreditation process, scheduled to begin in the fall of 1993, 
is the most significant accountability measure in PA25. The standards for accreditation are 
outcomes based and require schools to meet criteria within specific standards. 

Special education is not specifically cited in PA25, but special educators did participate 
in the development of the standards and agreed that accountability for programs serving students 
with disabilities could be measured within the general standards. As stated by SEA personnel, 
"It is now incumbent upon us in the field of special education to assure that such accountability 
is realized (to assure that student outcome data is disaggregated to document the performance of 
students in special education programs, for example)." 

The area of the model core curriculum provided for in PA25, however, is academically 
based and thus lacks accountability measures for some students with severe impairments. In 
addition, the third component of PA25, dealing with annual reporting, does not specifically 
mandate reports on the status of students with disabilities. Hence, this area of accountability is 
not mandated by the law and remains at the discretion of local districts. Similarly, there is no 
specific mandate for special educators to be on the school improvement teams required at both 
the building and district levels. The law does require that school improvement plans be 
established and the quality of these plans will be evaluated within the accreditation process. 

With the Michigan Educational Assessment Program, students with disabilities have 
traditionally been tested only if they are mainstreamed for the subject being taught. However, 
this is currently being challenged. With the advent of PA25, students will be assessed on core 
curriculum outcomes and expected to achieve a given standard of proficiency. Thus, issues have 
arisen around the appropriateness of a core curriculum for all students and whether schools need 
to develop other measures of success for some learners with disabilities. At this point, while the 
intent of PA25 is for accountability mechanisms to apply to all students, the true impact of the 
standards on special education is yet to be determined. 

In summary, Michigan special education is "on the same train" as regular eduction in the 
reform efforts of PA25. Due to the history of local autonomy, the State is extremely flexible in 
placing demands on local districts. Public pressure does play a role, however, as the publication 
of school-by-school results in local newspapers provides some measure of competition and 
accountability among school districts. 
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Conclusion and Recommendations 



A clear lesson emerging from the literature on educational accountability is that designing 
and implementing an effective accountability system is no easy matter. Part of the difficulty 
involves the politics of education with multiple, sometimes conflicting, agendas and a lack of a 
clear consensus on desired outcomes. Further complications arise over the measurement of 
progress toward proximate and ultimate outcomes, conducting cost benefit analysis of alternative 
approaches, and providing an equitable system of rewards and sanctions based on performance. 
The complex interrelationship of Federal, State, and local responsibilities often prevents a clear 
locus of accountability for desired outcomes. Local adaptations to even the most carefully 
constructed accountability system may lead to unanticipated consequences. 

In spite of these difficulties, it is clear that demands for accountability are not going to 
disappear. Special education is particularly vulnerable to criticism as State and local education 
agencies struggle with financial difficulties at the same time that costs rise for special education, 
particularly for "high visibility" instances of residential placement and related services. Special 
educators clearly need to consider demonstrating the results of programs and services for students 
demonstrating forms of accountability other than procedural compliance with statutes and 
regulations. Based on the analysis of issues of accountability presented in this report, a clear 
need exists for State level policymakers, researchers, practitioners, and other stakeholders to 
seriously address the issues surrounding the design and implementation of accountability systems. 
One of the difficulties cited in establishing effective accountability systems is the existence of 
multiple stakeholders, and a discussion of accountability systems needs to bring multiple 
constituencies "to the table" for the dialogue. An examination of the State accountability systems 
presented in this report also indicates that the discussion, as well as the subsequent design and 
implementation, of accountability systems must be a collaborative effort of general and special 
education. 

This report began with a definition of accountability as being subject to giving an account, 
to being answerable. Multiple parties are answerable for the nature and health of our educational 
system: government leaders, policymakers, administrators, teachers, parents, and the public at 
large. The next step in the process of examining and analyzing special education accountability 
is for general and special educators to work together to develop a comprehensive system of 
accountability for the effectiveness of educational programs for all students. 
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This inventory of state education reports was compiled by the Council of Chief State School 
Officers through a survey with all 57 state education agencies (50 states, D.C. , and 5 extra-state 
jurisdictions) that are members of the Council. The survey respondents were state representatives 
to the Council's Education Information Advisory Committee (EIAC). States had several 
opportunities to review and edit the information on their reports. Reports listed in the inventory 
are published or released by state education departments. In a few states, data collection and 
analysis are completed by state education departments and reports are released by local school 
districts. State education representatives determined if a given report has a purpose of education 
accountability, education indicators, or a report card on education in the state. 
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LEGISLATIVE UPDATE 
to the 

Chapter 1 Reform Network 
from 

The Independent Commission on Chapter 1 



Under the new law, funds will be allocated to schools on the basis of poverty. Schools which 
are successful in getting, their students to higher levels of proficiency will not lose their funding. 
They can be rewarded! . 

©KEY CHANGES IN - ACCOUNTABILITY 

The legislation improves accountability for result by requiring states to set high standards 
and to hold school districts and schools accountable for getting Title I students to meet state 
standards The Act is also requires each state to describe how it will help Title 1 schools and school 
> districts develop the capacity to provide students with a high quality education as described in the 
statute. 

A New Accountability Structure: 

) * States must define "adequate yearly progress" in a manner that results in ^continuous 

and substantial" yearly improvement of each local educational agency and school [that 
is] sufficient to achieve the goal of all children meeting higher levels of the State's 
performance standards.. particularly economically disadvantaged and limited English 
proficient children." Progress must be based primarily on student performance on 
§ state assessments, but other measures may also be used. 

* The state's plan must describe 3 levels of performance based on the assessments: 
"partially proficient, and proficient and advanced." 

& * States, school districts, and schools must report to the pubtic on the progress students 

make toward meeting state performance standards. Achievement data must be 
disaggregated by gender, by major racial and ethnic groups, by income, and by 
English proficiency status. 

h * School districts and schools have two years to move students toward meeting state 

proficiency standards. After providing technical assistance, states must take corrective 
action against school districts in which students do not make adequat "early progress 
toward meeting state performance standards. School districts mu: ' take similar 
corrective actions against schools. School districts and schools that succeed in 
assisting all students to meet state performance standards can be rewarded. 
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Is 'academic red-shirting' 
skewing the way we rank our schools? 



BY BILL ZLATOS 




egin with a tale of two schools — one educating children competently but winning little applause, another manip- 
ulating its testing program and garnering recognition and praise. One plays by the rules; the other plays the an- 
gles, perhaps even engaging deliberately in a dubious practice some might describe as "academic red-shirt- 
tog," a variation on the old sports gambit of holding players back until they strengthen and mature. The result 
yet one more crack in the fundamental validity of the test scores so many Americans rely r,n to measure the 
quality of education in this country. Take a closer look: 
Nestled in the Hudson River Valley near Albany, N.Y., is Castleton Elemer.tsry School, a brick castle of a school In a town 
of tho eamo name. Castletoa's students do well on standardised tests, but not well enough for state or federal officials to cre- 
ate a stir by presenting the school with any blue ribbons or plaques. Instead, Castleton's hallways sport the students' rendi- 
tions of Van Gogh's "Starry Night" alongside murals from "The Wizard of Oz" and "The Little Mermaid." 

Every year, teachers at Castleton flunk one or two of the 370 students at the school, and they do so with a sense of their 
own failing. (Last year, they were fortunate: They didn't flunk anyone.) Only a handful of children— five last year— are Identi- 
fied as disabled, and those children are taught in the regular classroom with the help of a special education teacher who 



B>U Zlatot it afree-ianct vrnttr basid in Pittsburgh. 



Opinion! expected by this maetant or iny o( iw juthum do not necewarlly reflect portions o( the NitioiuJ School Boird» Aiwcumon. 
24 NOVEMBER 1004 . _ THE AMERICAN SCHOOL BOARD JOURNAL 



DEC 19 '94 15:33 



BEST COPY AVAILABLE 
— Vtl \SI *H 

XdJ 6126240879 PAGE. 002 



DEC-19-94 HON 2:27 PM UNIVERSITY OF MINNESOTA 



FAX NO. 6126240879 



P. 3 



visits every room. The disabled students take the state-re- 
quired tests, and some of the youngsters pass the exams. 
"We actually get people out of special education," boasts 
Principal Dariene Adams. 

Not far away is Rvarton, an old mil! ;.»» filled with trailer 
parte, ranch houses, and Victorian mansions. Riv&rton is 
the pseudonym State University of New York researchers 
Richard Allington- and Anne McGill-Franzen gave to this 
New York community in their 1992 study. Bfcerten Elemen- 
tary School has received favorable newspaper coverage and 
a national award for educational excellence, based in part on 
ita high scores on the state's third-grade test 

Ninety-six percent of River ton's students pass the state 
fichisvement test But mat percentage was inflated, the re- 
searchers discovered: Half of the 68 students in kinder- 
garten through the second grade had bean flunked or had 
been placed in classes for handicapped students. Their 
scores weren't included in the district's average. If they had 
been, the researchers noted, the pass rate at Kiverton 
would have dropped from S8 percent to 78 percent 

Com nunities like these exist throughout the United 
States: Castletons that educate children without fanfare, 
and Rivertons whose teat scores and reputations get a boost 
from the decision to delay testing for low achievers— or do 
without it altogether. This practice is called exclusion, and it 
is the carnival mirror of education. It fattens the scores of 
some districts while making the figures of others look 
skimpy by comparison. 

. Last year, the MacArthur Foundation gave the Education 
Writers Association a grant to fund six national fellowships 
in education reporting. I received one of those fellowships 
to study how schools, knowingly or not inflate their test 
scores by excluding low-achieving students. Fart of my re- 
porting involved number-crunching: I obtained data on the 
testing practices of 14 of the largest school districts in the 
United States (see sidebar on page 26). Part involved visit- 
ing school districts and talking with a dminis trators, stu- 
dents, and testing experts across the country. 

What I found was that something as simple — and suppos- 
edly innocuous — as exclusion makes comparisons among 
schools, districts, and states virtually Impossible. Mix up 
the variation from state to state and district to district in the 
type of tests and norms used, the grade levels tested, the 
amount of time spent on test preparation, or the time of 
year die test I9 taken . Toss in fee difference in the number 
of students excluded, and test scores become a witch's 
brew of incongruous ingredients. Accountability vanishes. 

Yet the amount of state funding a school district receives 
can depend on test scores. A bond election can pass or foil 
on test scores, which the public often takes as a gauge of 
how well a school district is doing. Even a state takeover 
can depend in large part on whether a district's test scores 
riseorfatL 

Lauren Resnick. codlrector of the Learning Research and 
Development Center at the University of Pittsburgh, puts it 
succinctly. "The minute you allow exclusion," she says, 
"you open up a Pandora's box of manipulation designed to 
make fee school or district look as good as possible." 

Hitting pay dirt? 

Talk to school executives, parents, teachers, or even 
state education officials, and youU find two things: Schools 
can exclude students in many ways, and the practice is a 



pope _ way to Inflate test scores. Schools can exclude stu- 
dents' informally by suspending them or sending them to 
the gym or on field tips on test day. Or they can exclude 
students institutionally by funking them, placing them in 
transitional grades—an academic Umbo tantamount to 
flunking — or assigning them to classes for bilingual or dis- 
abled students. (Some parents of disabled children report 
forged signatures on individualized education plans feat say 
their children should be excluded from testing.) These chil- 
dren often are exempted from tests. If they are tested, their 
scores might ba left out of fee district's totals, which re- 
ceive mora scKitiny man classroom or grade scores. 

Like any good poison, exclusion is difficult to trace. The 
unsuspecting public and news media rarely ask questions 
about who is tested, and newspaper editors are unlikely to 
comu&a fee rankings of local schools with more numbers. 
Exclusion also provides a host of reasonable alibis: The 
school does not want to embarrass specific groups of chfl-* 
dran or make them anxious. The test was not normed is>v 
children like them. And besides, educators and policymak- 
ers haven't reached consensus on whether exclusion is ap- 
propriate — even ethical— in the first place. State regukV . 
tions and load policies usually allow it 

Schools, meanwhile, can enjoy a big payoff by failing to 
test everyone. The. more tow-achievers the school system 
excludes, the higher its test scores are likely to be. 
Whether a school benches a low-achieving student to inflate 
its scores or to spare fee youngster anxiety, the result ia the 
©me Test scores win rise. 

"You don't have to be much of a statistician to know if 
you can knock out [the bottom] 10 to 20 percent of your 
class, your scores are going to go up," says Thomas Hala- 
tiyna, an Arizona State University researcher who special- 
ises in the . ethics of testing practices. Robert Slavin. of 
Johns Hopkins university, estimates that flunking low- 
achieving students or placing them in transition grades in- 
flates scores by about 20 percentile points. After another 
year of covering fee same material, he says, these stu- 
dents probably will answer a few more questions right the 
second time they take the teat—yet their scores are com- 
pared to a national sample of younger students. 

Last year, fee state of Michigan inadvertently demon- 
strated what happens when you suddenly change fee rules 
about exclusion. For years, fee state department of educa- 
tion told school districts that if they did test students with 
disablHties or limited English proficiency, they could report 
fee results separately. Last year, though, the state education 
department broke from tradition and decided to include fee 
scores of bilingual and disabled students in its draft version 
of the state report card on fee performance of school dis- 
tricts; And once fee results of all students were combined, 
scores dropped. Districts bombarded fee department wife 
angry letters and phone calls, and fee state backed oft The 
data tapes were erased. As in previous years, Michigan is- 
sued a report card that excluded fee scores of handicapped 
and bilingual student*. • 

Or consider the variations found in the number of stu- 
dents tested in two large city school systems. Both Mas- 
sachusetts and Tennessee have regulations that allow the 
exclusion of some students. Massachusetts, like many 
states, says a child won't be tested if the child's iep says so 
or if the parents request that the child be excluded; Ten- 
nessee's regulations nay students will be tested unless the 



THE AMERICAN SCHOOL BOARD JOURNAL 



'94 15:34 



154 



6126240879 



NOVEMBER 1GB4 25 



PAGE. 003 



DEC- 19-94 WON 



2:28 PM UNIVERSITY OF MINNESOTA FAX NO. 6126240879 



P. 4 



ERIC * 



ep exclude them. In Tennessee, limited-English-speaking 
students may also be excluded. Yet, according to data pro- 
vided by die Boston cad Memphis school districts, only 66 
percent of (he students in Boston took fee district's reading 
achievement test in the 1992-93 school year, compared to 
93 percent of the students in Memphis.* . 

Why such a wide difference? For one thing, Mas- 
sachusetts uses a broad definition to identify students need- 
ing special education, thus increasing the pool of students 





alk interne carnival fun house and look in the mirror There's moredSSS- 

w 



tortioa than you might expect when it comes to school district tnsA-% 
scores. ' 'f 

Take the issue of "eligrMBty," for example. Most districts report test particapntior^ 
in terms of the number of "eligible" students. But eBgSbflity rules vary ajroughoa^^; 
the country, And few school districts make clear to the public •whom they're testzn]^l| 
and whom they're leaving out The Los Angeles Unified School District IsoEeoitK^ 
exceptions: It clearly describes who is tested, and who is exempted on the firstpagi^j 
of its testing report: More frequently, dMctsbtirymeirformsifin-- tfth^pn^eP 
It at afl. As a result, a naMos ever fond of rankings is increasing^ comparing 'ffisfeeteS 
and states met dont play by the ram© rules. And coiamunities assume meirfi^^sH 
schoolchildren are sticce^ding when perhaps they arent • 

The following data, based on testing reports end documents provided by 14 of fcfjii 
nation's school systems for the 1992-93 school year, show the kind of variations 
exist as weB as mo percentage of students who are represented in a school disnicf 
testkggrogram— and not simply those who are eligible. Two caveats are in order'to:/ 
avoid misunderstandings. First; for those school districts marked with an asteriskii^ 
hare provided estimates of the total number of students enrolled in the grades.; ^t-y' 
tested; such estimates are necessary because some school districts don't rep&£^|-;,^ 
their special education enrollments by specific grades but by ranges of grades (thliffsfe 
cumber of special education students in grades one through five, for example):$ecW : i: 
ond, districts test different grades, and these that test greater mmiberecihighi:^^, 
school students tend to have a lower percentage of students included in their scores^ 
because of high rates of absenteeism at the high school level Four of the schcot dis- 1 
in me list below—Baltimore, Philadelphia. XndianapoBs. and Miami— dM not uh 
dude high 6chool students in me testing programs cited here. • £ • • 

Finally, the cumbers of students tested are based on information the school cE&jgM 
tricts provided forme.testthey deemed moetnnportant'for accountability. \*J$&ijjm 



who could be exempted from testing. Boston appears to t 
exercising that option. According to data supplied by tr 
echool district, 21 percent of Boston's students are Identifk- 
a disabled— one of the largest proportions in the country- 
end 60 percent of me disabled students in Boston are eithe 
not tested or their scores are iaft out of the district tonus. A 
a result, one out of every three students in the Boston pubL' 
schools isn't represented in the school system's test score- 
Boston defends its practice, saying the test pool is repi\ . 

seatative of the cay's student poj. 
ulation. "If s based on the propo: 
tioa of students lis appropriate t 
have it based on," says MaryeEe 
Donahue, director of planning, n 
search, and development for th 
district Others insist the ga 
could have significant overall in' 
pact on scores. "Anyone i- 
Boston who believes that tes ' 
scores reported in daily paper. • 
accurately reflect student achieve 
meat in Boston is probably belli; 
given a rosier picture than what': 
true," says Anne "Wheelock, a for 
mer senior policy analyst for the 
Massachusetts Advocacy Center.! 
a public interest group for chil- 
dren. 

In either case, one thing is 
clean Given the differences ir. 
who takes the test* and how the . 
results are reported, you simply 
can't compare test scores is 
Boston with those in Memphis. 
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Who's left out 

Fine Lake Manor in Michigan 
is a microcosm of America's test- 
score mania. Residents of the 129 
homes in this upscale subdivision 
are part of the predominantly 
black and overwhelmingly poor 
Pontine School District Last year, 
they tried unsuccessfully to se- 
cede from Ponti&c and join pre- 
dominantly white and wealthy 
Bloomfield HHla. And much of the 
debate on the secession centered 
on whether test scores are a valid 
indicator of school performance. 
Pine Lake residents argued be- 
fore an administrative law judge 
that Bloomfield Hills offered a su- 
perior education. The proof: high 
test scores. One community 
member even testified that, tinder 
a divorce decree, his wife'9 chil- 
dren live with her ex-husband 

' For th« purpotot of this preji* t examined 
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Some schools succumb 

to the temptation to 
make their scores look, 
artificially good 



during the weak because the ex-husband lives in a better 
school district 

Given these stakes, Wa not surprising some schools suc- 
cumb to the temptation to make their scores look artificially 
good. Of course, districts rarer/ broadcast the fact that they 
exclude students from testing. And they even more rarely 
confess they've done so to raise test scores. Still, plenty of 
evidence suggests children are being benched from tests 
throughout (he United States. 

According to the National Center on Educational Out- 
comes at the University of Minnesota, nearly half of all dis- 
abled students is the United States are excluded from the 
National Assessment of Educational Progress (naep), the 
U.S. yardstick for evaluating education. In addition, a 1393 
study commissioned for NAB? shows 
wide disparities in the number of dis- 
abled students tested from state to 
state. According to the report, Wash- 
iagton, D .C, was least Ekely to test dis- 
abled students, and California was 
least likely to test those with limited 
EngHah skills. 

The report farther estimated how 
state rankings might change if all dis- 
abled and bfSngaal students had been 
tested and had scored lower than other 
students. Among the findings: North 
Carolina would have jumped nine 
places on eighth-grade math, and 
Texas would have dropped she places 
on Its fourth-grade scores. 

Often, too, fee number of disabled 
students tested will vary between 
school districts in the same state. Last 
year, for example, a survey released by the Massachusetts 
Deportment of Education showed that some school districts 
tested none of their disabled students and others tested all 
of them. Among the reasons given for the exclusion: the 
students' emotional or attention omits, their limited cogni- 
tive abilities, and their parents' requests. The researchers 
offered another reason as weH: "The desire to look good Is, 
perhaps, the underlying single biggest reason why it is eas- 
ier to exclude students with special needs from group test- 
ing than to include them,* they said. 

Statistics from the National Center on Education Out- 
comes paint & similar picture nationwide. When the center 
looked at the number of students taking state tests, it found 
mat test participation by handicapped students ranged from 
2 percent in Michigan to 100 percent in Delaware. James 
Yaeldyke. director of the center, says school administrators 
STmtamey are pressured by their state and the commu- 
nity to look good, so they test as few handicapped students 
as possible: 

Bilingual students fare the same. According to a 1992 
study by researchers Tony C. M. Lam of the University of 
Toronto and Wayne I. Gordon of the University of New 
Mexico, most states have no policies on exempting bilin- 
gual students from testing. Of the 17 states that do. the poli- 
cies range from permitting no exemptions to excluding stu. 
dents for up to three years, leading the authors to conclude 
that decisions on whether bilingual students get tested are 
"often rather arbitrary." 

Whether the students are identified as being limited in 



S J, ' 
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English proficiency or in need of special education, exdud- 
iag them from testing helps make a district look good. 

Take the cose of the Oklahoma City Public Schools. Dur- . 
tag the late '80s and eorty '80s, Oklahoma City exempted 
thousands of handicapped and limited English students 
from state and local tests and placed thousands of others in 
transition classes. 

The result of this exclusion vraa twofold: First, test scores 
mushroomed, rising from 39 percent of the students scor- 
ing above the national norm la 1987*88 to S3 percent in 
1991-92. Second, the number of students who were tested 
dwindled from 34,000 in 12 grades to. 19,000 b 10 grades. 
The school district's <S,0GCHstudent drop in enrollment dur- 
ing the same period doesn't explain die decline. 

The Oklahoma City Schools gained 
a lot of mileage from the improved 
scores. The number of schools on die 
state's probationary Hat for low test 
scores dropped from 21 to one. A fed- 
eral court dismissed a desegregation 
lawsuit in part because of the rising 
scores of black students. And the dis- 
trict received a Leadership In Learn- 
ing Award from the American Associ- 
ation of School Administrators for die 
district's effective schools. The proof 
of the school system's success? Ris- 
ing test scores. 

Oklahoma City's practices were by 
ho means unique. Three years ago, 
■ Forest Rsece, then a member of the 
state Board of Education in Okla- 
homa, surveyed 20 schools that had 
worked their way off die state's proba- 
tionary list and found that twoof ever y three students in 
these schools had been exempteorirom Gtate~tests because 
of a disabuhy or a problem speaking English. 

Oklahoma City Superintendent Betty Mason, who as- 
sumed the post in 1992, sums up the state's high-stakes 
atmosphere that pits superintendent against superinten- 
dent the way athletic rankings pit ccach against coach. 
Grabbing a newspaper from her desk and holding ft up in 
the air, she describes die rivalry the dsy that tests scores 
are released. "When you open that paper In the morning 
and when you see how Tulsa compares to Oklahoma City, 
you want to be on top of that list," she Gays. 

Under Mason, the Oklahoma City school board has re- 
laxed its policy on retaining students In grade and elimi- 
nated seven transition grades. One transition grade and 
widespread exemptions remain. And die rise in test scores 
apparendy has slowed: Last year, only one grade out of five 
showed an increase in its test scores. 

Why test feem an? 

Make no mistake about it Most educators are not ex- 
empting students simply to boost test scores. Arguments 
about the appropriateness of standardized te9ts for some 
students bear weight. A 1987 study conducted by experts in 
the field of special education and cited in the magazine fir- 
ceptionol Childrtsn found that the developers of many of the 
27 most popular standardized tests failed to provide evi- 
dence that their tests were valid for disabled students. 

But as Resnlok of Pittsburgh and AHington of the State 
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University of New York point out, there's an inherent dan- 
ger In excluding kids: If children aren't tested, they be- 
come invisible, like other critics of exclusion, both con- 
sider the testing of all students a key component of 
improving education for these students, and both favor test- 
ing accommodations, such as extra time, large-print or 
Braille versions^ translators, and tests in the students' na- 
tive languages when appropriate, Resnick'a work as codi- 
rector of the New Standards Project in fact involves devel- 
oping a series of testa, based on world-class standards, for 
students in math, English, science, and other subjects. Un- 
dergirding the project is the philosophy that all students 
m& be tested. 

Others worry feat allowing schools to exclude some stu- 
dents is inherently discriminatory as 
welL "It tends to have a heavy redd 
and poverty piece attached to ft." says . 
Josn First, executive codirector of me 
Boston-based National CoaBtion of Ad- 
vocates for Students. "The kids most 
Hkely to be excluded [for disciplinary 
reasons] are kids who are poor or 
members of racial or language mtaori- 
• ties." 

Most important, contends Paul 
LeMshieu. director of fee Delaware 
Education Research and Development 
Center, exclusion institutionalises bad 
education practice. Students who are 
flunked or placed in transition grades 
often endure another year of fee same 
material taught by fee same teacher 
using fee same unsuccessful methods. 
Depending on fee quality of their spe- 
i del education or bilingual classes* 
these students might stay behind for 
yeare. 1 don't it's premeditated 
murder," LeMahieu says. "It may well be murder, though." 

.Alicia Sosa, director of fee Center for Test Equity at the 
Intercultural Development Research Association, agrees. 
She doesn't want bilingual students to drown in an ocean of 
) testa, but she also doesnt want them shunted into the wad- 
ing pool for fee remainder of their academic careers. She 
and ether advocates for bilingual education worry that fail- 
ing to test students wife a limited command of English de- 
prives policymakers of information on youngsters who are 
most likely to drop out 
Resources are an issue as well "All things being equal, 
5 fee eyotem is likely to send more quality teachers to the 
kids who get tested," says Kenji Hakuta, an education pro- 
fessor at Stanford University and chairman of a panel urging 
fee overhaul of bilingual education. 

To guard against that possibility, some school districts 
and states have made it difficult to omit students from test- 
. ing. Sixteen states and 8k districts take part in the New 
9 Standards Project Codirector Resnick says the project will 
evaluate students on real-life tasks, taking minutes or 
months, performed alone or in groups. The project will test 
students In Spanish and probably other languages, and stu- 
dents wife learning disabilities and mental and emotional 
handicaps will be tested, too, "If you're not forced to include 
k them in the tasting, then you're not forced to include them 
O in fee instruction," sayo Resnick 
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New York City and Miami also have policies that encour- 
age widespread leAung. New York includes in its totals the 
scores of all handicapped students in resource rooms, re- 
gardless oi how much time they spend there; fee city also 
offers a math in four languages—English, Spanish, Chi- 
nese, and Haitian-Creole. Miami audits its more than 300 
schools to ensure that every student is accounted for on fee 
tests— and that no student is tested or exempted who's not 
supposed to 'be. Such oversight makes it difficult for indMd- 
ual schools to push fee envelope on whom they're not test- 
ing; fee central office and fee school board are always 
watching. 

As part of its education reforms, Kentucky has adopted 
the toughest testing standards in fee nation. Everyone seek- 
ing a high school diploma must be 
tested, though students wife problems 
speaking English arsy receive a quali- 
fied, one-year exemption, and moder- 
ately or severely handicapped students 
may take an alternative test Further- 
more, students who are not tested, au- 
tomatically receive a "novice" rating, 
fee lowest level of proficiency, says 
Scott Trimble, state director of assess- 
ment implementation, and that rating 
figures in fee school system's score. In 
Kentucky, too.test scores are used for 
such high-stakes decisions as state 
takeovers. 

Moreover, states like Michigan and 
New York report the scores of regular 
students and handicapped or bilingual 
students separately. Yseldvk^ of Min- 
nesota says that's tHe least states 
should do. That way, he says, schools 
and states have some information on 
how those children are doing. 
Allington. of fee State University of New York, disagrees. 
"It continues to perpetuate fee second-class citizen status of 
handicapped students," he says, addingthat people pay 
more attention to fee other scores. 

The most sweeping suggestion for cleaning up testing 
abuses comes from Boston College testing expert George 
Madaus. who recommends creating a national board to 
monitor testing practices. That board, Madaus says, could 
not only monitor any national exams feat might be devel- 
oped but could also monitor fee disclosure of who is tested. 
Bruce D. Spencer, of the Methodology Research Center at 
Northwestern University, wants districts to report fee per- 
centage of all students fee district tests. "If s all couched in 
bureaucratic criteria as to who's eligible and who's not" he 
complains. "It's hard to know as a citizen or as a sodal sci- 
entist exactly who is being excluded." 

For children who are excluded, being left out of tests 
starts a evele of bad education feat ultimately ends in fail- 
ure. "The school doesn't feel responsible for getting those 
excluded kids to high levels of achievement" says Kati Hay- 
cock, director of the Washington, D.C.-based Education 
Trust "In the end. we tell fee world they're not achieving 
because they're dumb, their parents don't care, [or] they 
dont have books at home. We tell fee world all these rea- 
sons, when in truth the reason they dont do well is because 
w| dji^educate them." 
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An accountable 
education system is 
one which ensures that 
all children, including 
those with disabilities, 
benefit from their 
educational experience 
through equal access, . 
high standards, and high 
expectations. Through 
these experiences, all 
children will become 
"caring, productive, 
socially involved citizens 
who are committed to 
lifelong learning." 

Quotation from the mission 
statement of the National Agenda 
for Achieving Better Results for 
Children and Youth with 
Disabilities. June, 1994. 
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APPENDIX C: Model Diagrams 



Report of Accountability Focus Group Meeting 
Project FORUM at NASDSE, Task U7-3-4 



Page 14 
March 21, 1995 
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standards,. 

Continuous improvement 

• Curriculum 
Baseline data 
Staff ievelopment 
Ongoing^rtieasurement 

• Longitudinal: studies 
° Multiple indicators (dropouts, 



luitipie measures of system 
effectiveness-cognitive and 
noncognitive 

Flexibility and accessibility to 

program 

Blended system 

Appropriate reinforcements 

(sanctions and incentives) 

School report card 

MIS 
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e Academic/nonacademic 

standards 
° Individual expectations 

(cognitive and noncognitive) 

• Multiple measures of individual 
student progress 

° Flexibility in assessment 
modes 

° Continuous progress with 
benchmarks along the way 

• Parent involvement 

° Teacher empowerment 
o |EP 
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• IEP 

• LRE 

• Access standards 

• Procedural safeguards 
° Parent involvement 

° Staffing credentials 

° Finance 

° Staff development 

s Policy development 

° Diversity 

• IDEA 
° MIS 

° Demand for use of 
effective practice 
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Examples 
° Standards 

• Continuous improvement 

• Curriculum 

° Baseline data 
0 Staff development 

• Ongoing measurement 

• Longitudinal studies 

° Multiple indicators (dropouts, 
retention, and completion) 

• Multiple measures of system 
effectiveness-cognitive and 
noncognitive 

° Flexibility and accessibility to 

program 
« Blended system 

• Appropriate reinforcements 
(sanctions and incentives) 

• School report card 
MIS 




Examples 

• IEP 

• LRE 

• Access standards 

• Procedural safeguards 
o Parent involvement 

• Staffing credentials 

• Finance 

« Staff development 

• Policy development 

• Diversity 

• IDEA 

• MIS 

» Demand for use of effective 
practice 



Examples 

• Academic/nonacademic standards 

• Individual expectations 
(cognitive and noncognitive) 

0 Multiple measures of individual student 
progress 

• Flexibility in assessment modes 

• Continuous progress with benchmarks 
along the way 

0 Parent involvement 
0 Teacher empowerment 

• IEP 
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Shapes inputs required 
to achieve results. 
Demands constant 
renewal of inputs/process 
measures. 

Ensures continuity and 
consistency of inputs 
across populations 




© Demands high 

expectations for teaching. 
L Requires connection 
between what we know 
about a child and 
instructional strategies. 
S Strengthens emphasis on 
individual learning. 
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system results. 
S Validates system 
standards. 
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Provides reliable, 
valid data that informs 
system configuration. 
Defines boundaries 
for opportunity/ 
flexibility. 

Reinforces/renews 
resources for 
systems. 




C Establishes tools 
necessary to support 
and facilitate learning. 

L Requires opportunity, 
for all/each. 

S Strengthens 
emphasis on 
individualization and 
parent involvement. 
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